A Bilingual Benchmark for Evaluating Diagnostic Performance of Multimodal Large Language Models in Radiology (RadM-Bench): Evaluation Development and Validation

Qingxia Wu   ^{1 *} , MD ;   Qingxia Wu   ^{2 *} , PhD ;   Peipei Zhang   ^{3 *} , MM ;   Zhifeng Yi   ⁴ , MM ;   Yu Shen   ¹ , PhD ;   Yan Bai   ¹ , MD ;   Hongna Tan   ¹ , MD ;   Pei Dong   ² , PhD ;   Zhong Xue   ^{2, 5} , PhD ;   Neil Roberts   ^{6, 7} , PhD ;   Meiyun Wang   ¹ , PhD, MD

¹ Department of Medical Imaging, Henan Provincial People’s Hospital & People’s Hospital of Zhengzhou University, Zhengzhou, Henan, China

² Department of Research and Cooperation, United Imaging Intelligence, Beijing, Beijing, China

³ Department of Nuclear Medicine, Hangzhou Cancer Hospital, Hangzhou, Zhejiang, China

⁴ Emergency Department, Huangshi Central Hospital, Huangshi, Hubei, China

⁵ Department of Research and Cooperation, United Imaging Intelligence, Shanghai, Shanghai, China

⁶ Biomedical Research Institute, Henan Academy of Sciences, Zhengzhou, Henan, China

⁷ Center for Reproductive Health, Institute for Regeneration and Repair, University of Edinburgh, Edinburgh, Scotland, United Kingdom

*these authors contributed equally