Performance of Large Language Models in Numerical Versus Semantic Medical Knowledge: Cross-Sectional Benchmarking Study on Evidence-Based Questions and Answers

Eden Avnat   ^{1, 2} , MPH, MD ;   Michal Levy   ^{3, 4} , BCS, MD ;   Daniel Herstain   ¹ , MD ;   Elia Yanko   ⁵ , BSc ;   Daniel Ben Joya   ^{2, 6} , MD ;   Michal Tzuchman Katz   ² , MD ;   Dafna Eshel   ² , MD ;   Sahar Laros   ^{1, 2} , BMedSci ;   Yael Dagan   ^{1, 2} , BMedSci ;   Shahar Barami   ^{1, 2} , BMedSci ;   Joseph Mermelstein   ² , BCS ;   Shahar Ovadia   ² , MCS ;   Noam Shomron   ¹ , PhD ;   Varda Shalev   ¹ , MD, MPH ;   Raja-Elie E Abdulnour   ⁷ , MD

¹ Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel

² Kahun Medical Ltd, Givatayim, Israel

³ Faculty of Medicine, Hebrew University of Jerusalem, Jerusalem, Israel

⁴ School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel

⁵ The Azrieli Faculty of Medicine, Bar-Ilan University, Safed, Israel

⁶ Kaplan Medical Center, Rehovot, Israel

⁷ Division of Pulmonary and Critical Care Medicine, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, United States