Token Probabilities to Mitigate Large Language Models Overconfidence in Answering Medical Questions: Quantitative Study

Raphaël Bentegeac   ^{1, 2 *} , MS, MD ;   Bastien Le Guellec   ^{3, 4 *} , MS, MD ;   Grégory Kuchcinski   ^{3, 4} , MD, PhD ;   Philippe Amouyel   ^{1, 2, 5} , MD, PhD ;   Aghiles Hamroun   ^{1, 2, 5} , MD, PhD

¹ Department of Public Health, Lille University, Lille University Hospital Center, Lille, France

² Univ. Lille, Inserm, Centre Hosp. Univ Lille, Institut Pasteur de Lille, UMR1167—Labex DISTALZ—RID-AGE—Risk Factors and Molecular Determinants of Aging-Related Diseases, Lille, France

³ Department of Neuroradiology, Lille University, Lille University Hospital Center, Lille, France

⁴ Univ. Lille, Inserm, CHU Lille, U1172—LilNCog—Lille Neuroscience & Cognition, Lille, France

⁵ Univ. Lille, CNRS, Inserm, CHU Lille, Institut Pasteur de Lille, US 41 - UAR 2014 - PLBS, Lille, France

*these authors contributed equally