Published on in Vol 27 (2025)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/70901, first published .
Beyond Benchmarks: Evaluating Generalist Medical Artificial Intelligence With Psychometrics

Beyond Benchmarks: Evaluating Generalist Medical Artificial Intelligence With Psychometrics

Beyond Benchmarks: Evaluating Generalist Medical Artificial Intelligence With Psychometrics

Luning Sun   1 , PhD ;   Christopher Gibbons   2 , PhD ;   José Hernández-Orallo   3, 4 , PhD ;   Xiting Wang   5 , PhD ;   Liming Jiang   6 , MSc ;   David Stillwell   1 , PhD ;   Fang Luo   6 * , PhD ;   Xing Xie   7 * , PhD

1 The Psychometrics Centre, Cambridge Judge Business School, University of Cambridge, Cambridge, United Kingdom

2 Oracle Health, Austin, TX, United States

3 Valencian Research Institute for Artificial Intelligence (VRAIN), Universitat Politècnica de València, València, Spain

4 Valencian Graduate School and Research Network of AI, València, Spain

5 Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China

6 Faculty of Psychology, Beijing Normal University, Beijing, China

7 Microsoft Research Asia (China), Beijing, China

*these authors contributed equally

Corresponding Author: