This paper is in the following e-collection/theme issue:

Generative Language Models Including ChatGPT (678) AI Language Models in Health Care (190) Learning and Education (107) Artificial Intelligence (2080) Applications of AI (274)

Published on in Vol 27 (2025)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/65146, first published .
Unveiling GPT-4V's hidden challenges behind high accuracy on USMLE questions: Observational Study

Unveiling GPT-4V's hidden challenges behind high accuracy on USMLE questions: Observational Study

Unveiling GPT-4V's hidden challenges behind high accuracy on USMLE questions: Observational Study

Authors of this article:

Zhichao Yang1 Author Orcid Image ;   Zonghai Yao1 Author Orcid Image ;   Mahbuba Tasmin1 Author Orcid Image ;   Parth Vashisht1 Author Orcid Image ;   Won Seok Jang2 Author Orcid Image ;   Feiyun Ouyang2 Author Orcid Image ;   Beining Wang3 Author Orcid Image ;   David McManus4 Author Orcid Image ;   Dan Berlowitz5, 6 Author Orcid Image ;   Hong Yu1, 2, 6, 7 Author Orcid Image
Article Authors Cited by (11) Tweetations Metrics

Journals

  1. Kim S, Wihl J, Schramm S, Berberich C, Rosenkranz E, Schmitzer L, Serguen K, Klenk C, Lenhart N, Zimmer C, Wiestler B, Hedderich D. Human-AI collaboration in large language model-assisted brain MRI differential diagnosis: a usability study. European Radiology 2025;35(9):5252 View
  2. Azizoglu M, Escolino M, Kamci T, Klyuev S, Perez Bertolez S, Risteski T, Elhalaby I, Borkar N, Esposito C, Okur M, Lacher M, Mutanen A, Shehata S, Chiarenza F, Davenport M. Generative Artificial Intelligence Accuracy in Interpreting Forest Plots in Pediatric Surgery Meta-analyses: A Perspective From Pediatric Surgery Meta-analysis Study Group (PESMA). Journal of Pediatric Surgery 2025;60(7):162359 View
  3. Zhou S, Xu Z, Zhang M, Xu C, Guo Y, Zhan Z, Fang Y, Ding S, Wang J, Xu K, Xia L, Yeung J, Zha D, Cai D, Melton G, Lin M, Zhang R. Large language models for disease diagnosis: a scoping review. npj Artificial Intelligence 2025;1(1) View
  4. Boltaboyeva A, Baigarayeva Z, Imanbek B, Ozhikenov K, Getahun A, Aidarova T, Karymsakova N. A Review of Innovative Medical Rehabilitation Systems with Scalable AI-Assisted Platforms for Sensor-Based Recovery Monitoring. Applied Sciences 2025;15(12):6840 View
  5. Mavrych V, Yousef E, Yaqinuddin A, Bolgova O. Large language models in medical education: a comparative cross-platform evaluation in answering histological questions. Medical Education Online 2025;30(1) View
  6. Othman A, Sharqawi A, MohammedAziz A, Ali W, Alatiyyah A, Mirah M. Assessing the Accuracy and Completeness of AI-Generated Dental Responses: An Evaluation of the Chat-GPT Model. Healthcare 2025;13(17):2144 View
  7. Jaleel A, Aziz U, Farid G, Zahid Bashir M, Mirza T, Khizar Abbas S, Aslam S, Sikander R. Evaluating the Potential and Accuracy of ChatGPT-3.5 and 4.0 in Medical Licensing and In-Training Examinations: Systematic Review and Meta-Analysis. JMIR Medical Education 2025;11:e68070 View
  8. Kasagga A, Sapkota A, Changaramkumarath G, Abucha J, Wollel M, Somannagari N, Husami M, Hailu K, Kasagga E. Performance of ChatGPT and Large Language Models on Medical Licensing Exams Worldwide: A Systematic Review and Network Meta-Analysis With Meta-Regression. Cureus 2025 View
  9. Okayo O, Panwal N, Ihuarulam O, Nzunde M, Nyimwadang F, Oladosu T, Osunde. A. Transforming Medical Laboratory Science with Vision-Language Models: A Focus on Microscopy in Microbiology, Hematology, and Histopathology. Oncology, Nuclear Medicine and Transplantology 2025;1(2):onmt008 View
  10. Nguyen V, Vuong T, Nguyen V, Ma, H. Benchmarking large-language-model vision capabilities in oral and maxillofacial anatomy: A cross-sectional study. PLOS One 2025;20(10):e0335775 View

Conference Proceedings

  1. Lv Y, Yu Q, Wang Z, Liang Y, Wang F, Li S. 2025 7th International Conference on Artificial Intelligence Technologies and Applications (ICAITA). CoupletEval: A Novel Benchmark for Assessing Chinese Linguistic Proficiency in Large Language Models View