This paper is in the following e-collection/theme issue:

Generative Language Models Including ChatGPT (708) Natural Language Processing (917) Clinical Communication, Electronic Consultation and Telehealth (758) Chatbots and Conversational Agents (740) E-Health / Health Services Research and New Models of Care (718) Artificial Intelligence (2129) Decision Support for Health Professionals (1539)

Published on in Vol 26 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/53297, first published .
Triage Performance Across Large Language Models, ChatGPT, and Untrained Doctors in Emergency Medicine: Comparative Study

Triage Performance Across Large Language Models, ChatGPT, and Untrained Doctors in Emergency Medicine: Comparative Study

Triage Performance Across Large Language Models, ChatGPT, and Untrained Doctors in Emergency Medicine: Comparative Study

Authors of this article:

Lars Masanneck1, 2 Author Orcid Image ;   Linea Schmidt2 Author Orcid Image ;   Antonia Seifert3 Author Orcid Image ;   Tristan Kölsche1 Author Orcid Image ;   Niklas Huntemann1 Author Orcid Image ;   Robin Jansen1 Author Orcid Image ;   Mohammed Mehsin1 Author Orcid Image ;   Michael Bernhard3 Author Orcid Image ;   Sven G Meuth1 Author Orcid Image ;   Lennert Böhm3 Author Orcid Image ;   Marc Pawlitzki1 Author Orcid Image
Article Authors Cited by (44) Tweetations (6) Metrics

Journals

  1. Armbruster J, Bussmann F, Rothhaas C, Titze N, Grützner P, Freischmidt H. “Doctor ChatGPT, Can You Help Me?” The Patient’s Perspective: Cross-Sectional Study. Journal of Medical Internet Research 2024;26:e58831 View
  2. Le K, Chen J, Mai D, Le K. An Evaluation on the Potential of Large Language Models for Use in Trauma Triage. Emergency Care and Medicine 2024;1(4):350 View
  3. Sorich M, Mangoni A, Bacchi S, Menz B, Hopkins A. The Triage and Diagnostic Accuracy of Frontier Large Language Models: Updated Comparison to Physician Performance. Journal of Medical Internet Research 2024;26:e67409 View
  4. Geneş M, Deveci B. A Clinical Evaluation of Cardiovascular Emergencies: A Comparison of Responses from ChatGPT, Emergency Physicians, and Cardiologists. Diagnostics 2024;14(23):2731 View
  5. Miralles Linares F, Martín Quirós A, Jaén Cañadas M. Triaje en urgencias basado en inteligencia artificial: una herramienta prometedora. Medicina Clínica 2025;164(8):437 View
  6. Zhang P, Shi J, Kamel Boulos M. Generative AI in Medicine and Healthcare: Moving Beyond the ‘Peak of Inflated Expectations’. Future Internet 2024;16(12):462 View
  7. Arslan B, Nuhoglu C, Satici M, Altinbilek E. Evaluating LLM-based generative AI tools in emergency triage: A comparative study of ChatGPT Plus, Copilot Pro, and triage nurses. The American Journal of Emergency Medicine 2025;89:174 View
  8. Chang Y, Lin Y, Huang F, Chen D, Chung Y, Chen W, Wang C. Using machine learning and natural language processing in triage for prediction of clinical disposition in the emergency department. BMC Emergency Medicine 2024;24(1) View
  9. Koirala P, Thongprayoon C, Miao J, Garcia Valencia O, Sheikh M, Suppadungsuk S, Mao M, Pham J, Craici I, Cheungpasitporn W. Evaluating AI performance in nephrology triage and subspecialty referrals. Scientific Reports 2025;15(1) View
  10. Masanneck L, Meuth S, Pawlitzki M. Evaluating base and retrieval augmented LLMs with document or online support for evidence based neurology. npj Digital Medicine 2025;8(1) View
  11. Rider N, Li Y, Chin A, DiGiacomo D, Dutmer C, Farmer J, Roberts K, Savova G, Ong M. Evaluating large language model performance to support the diagnosis and management of patients with primary immune disorders. Journal of Allergy and Clinical Immunology 2025;156(1):81 View
  12. Zaboli A, Brigo F, Magnarelli G, Gorick H, Garbin T, Clauser P, Sibilio S, Brigiari G, Massar M, Mian M, Pfeifer N, Turcato G. Reproducibility of the Manchester Triage System: a multicentre vignette study. Emergency Medicine Journal 2025;42(6):403 View
  13. Miralles Linares F, Martín Quirós A, Jaén Cañadas M. Artificial intelligence-based triage in emergency departments: A promising tool. Medicina Clínica (English Edition) 2025;164(8):437 View
  14. Zaboli A, Brigo F, Brigiari G, Massar M, Parodi M, Pfeifer N, Magnarelli G, Turcato G. Chat-GPT in triage: Still far from surpassing human expertise - An observational study. The American Journal of Emergency Medicine 2025;92:165 View
  15. Wang J, Shue K, Liu L, Hu G. Preliminary evaluation of ChatGPT model iterations in emergency department diagnostics. Scientific Reports 2025;15(1) View
  16. Suppan M, Fubini P, Stefani A, Gisselbaek M, Samer C, Savoldelli G. Performance of 3 Conversational Generative Artificial Intelligence Models for Computing Maximum Safe Doses of Local Anesthetics: Comparative Analysis. JMIR AI 2025;4:e66796 View
  17. Meyer N, Meyer J. A Practical Guide to the Utilization of ChatGPT in the Emergency Department: A Systematic Review of Current Applications, Future Directions, and Limitations. Cureus 2025 View
  18. Zakirah A, Saniasiaya J. Parental Perception on Usage of AI Chatbot to Understand Paediatric Otorhinolaryngology Condition: A Survey. Indian Journal of Otolaryngology and Head & Neck Surgery 2025;77(5):2078 View
  19. Liu S, Wright A, McCoy A, Huang S, Steitz B, Wright A. Detecting emergencies in patient portal messages using large language models and knowledge graph-based retrieval-augmented generation. Journal of the American Medical Informatics Association 2025;32(6):1032 View
  20. Benavent D, Madrid-García A. Large language models and rheumatology: are we there yet?. Rheumatology Advances in Practice 2025;9(2) View
  21. Zou Y, Ye R, Gao Y, Zhou J, Li Y, Chen W, Zha F, Wang Y. Comparison of triage performance among DRP tool, ChatGPT, and outpatient rehabilitation doctors. Scientific Reports 2025;15(1) View
  22. Luo X, Tham Y, Giuffrè M, Ranisch R, Daher M, Lam K, Eriksen A, Hsu C, Ozaki A, Moraes F, Khanna S, Su K, Begagić E, Bian Z, Chen Y, Estill J. Reporting guideline for the use of Generative Artificial intelligence tools in MEdical Research: the GAMER Statement. BMJ Evidence-Based Medicine 2025;30(6):390 View
  23. Vrdoljak J, Boban Z, Males I, Skrabic R, Kumric M, Ottosen A, Clemencau A, Bozic J, Völker S. Evaluating large language and large reasoning models as decision support tools in emergency internal medicine. Computers in Biology and Medicine 2025;192:110351 View
  24. Choudhury A, Shahsavar Y, Shamszare H. User Intent to Use DeepSeek for Health Care Purposes and Their Trust in the Large Language Model: Multinational Survey Study. JMIR Human Factors 2025;12:e72867 View
  25. Hains L, Kovoor J, Stretton B, Gupta A, Zaka A, Carmichael G, Kefalianos J, Ei W, Leslie A, Booth A, Satheakeerthy S, Beath A, Arafat Y, Jacob M, Bruening M, Chan W, Bacchi S. When One Size Does not Fit All—Artificial Intelligence in Australian Rural Health. Australian Journal of Rural Health 2025;33(3) View
  26. Pan W, Zhang S, Wang Y, Quan Z, Zhu Y, Fang Z, Yang X. Clinical Management of Wasp Stings Using Large Language Models: Cross-Sectional Evaluation Study. Journal of Medical Internet Research 2025;27:e67489 View
  27. Eminovic S, Levita B, Dell’Orco A, Leppig J, Nawabi J, Penzkofer T. Comparison of Multiple State-of-the-Art Large Language Models for Patient Education Prior to CT and MRI Examinations. Journal of Personalized Medicine 2025;15(6):235 View
  28. Wu G, Yang F. Navigating the Transformative Impact of Artificial Intelligence in Health Services Research. Health Science Reports 2025;8(6) View
  29. Nguyen D, Lee S, Synghal R, Chan L, Justus F, Moromisato M, Shao T, Wang C, Kellogg M, Anwar B, Creech J, Ochoa K, Gigliotti D, Ortiz O, La K, Nguyen K. Digital transformation with clinical alerts and personalized care systems in an integrated value based model. npj Digital Medicine 2025;8(1) View
  30. Masayoshi K, Hashimoto M, Toda N, Mori H, Kobayashi G, Haque H, So M, Jinzaki M. Training Language Models for Estimating Priority Levels in Ultrasound Examination Waitlists: Algorithm Development and Validation. JMIR AI 2025;4:e68020 View
  31. Jagatheesaperumal S, Pandiyarajan A, Boopathy P, Deepa N, Barreto A, de Albuquerque V. A review on recent advancements of ChatGPT and datafication in healthcare applications. Computers in Biology and Medicine 2025;197:110885 View
  32. Günay Polatkan Ş, Sığırlı D, Durak V, Alak Ç, Kan I. Performance of Generative AI Models on Cardiology Practice in Emergency Service: A Pilot Evaluation of GPT-4.o and Gemini-1.5-Flash. Uludağ Üniversitesi Tıp Fakültesi Dergisi 2025;51(2):239 View
  33. Neary C, Ong M, Farmer J. Sociodemographic drivers of care disparity among patients with inborn errors of immunity. Current Opinion in Pediatrics 2025;37(6):613 View
  34. Masanneck L, Epping P, Meuth S, Pawlitzki M. Evaluating Web Retrieval–Assisted Large Language Models With and Without Whitelisting for Evidence-Based Neurology: Comparative Study. Journal of Medical Internet Research 2025;27:e79379 View
  35. Yıldırım C, Aykut A, Günsoy E, Öncül M. Evaluating GPT-4o for emergency disposition of complex respiratory cases with pulmonology consultation: a diagnostic accuracy study. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine 2025;33(1) View
  36. Peng X, Lu Q, Liu K. Knowledge Breadth and Depth Measurement of Large Language Models (LLMs). Proceedings of the Association for Information Science and Technology 2025;62(1):495 View
  37. Levita B, Eminovic S, Lüdemann W, Schnapauff D, Schmidt R, Haack A, Dell’Orco A, Nawabi J, Penzkofer T. Large language models for patient education prior to interventional radiology procedures: a comparative study. CVIR Endovascular 2025;8(1) View
  38. Xu L, Zhao W, Huang X. Diagnosis and Triage Performance of Contemporary Large Language Models on Short Clinical Vignettes. Journal of Medical Systems 2025;49(1) View
  39. Bennett I, Matyas B. Artificial Intelligence in Public Health. Primary Care: Clinics in Office Practice 2025 View
  40. Alomari L, Alshammari M, Arbaeen A, Alshehri R, Almalki H. Safety and accuracy of AI in triaging patients in the emergency department. International Journal of Emergency Medicine 2025;18(1) View
  41. Cohen L, Gupta A, Patel P, Gill G, Bains H, Gupta S. The Role of Large Language Models in Ophthalmology: A Review of Current Applications, Performance, and Future Directions. Cureus 2025 View
  42. Wasserman G, Grinberg N, Peleg O, Shuster A, Ianculovici C, Hurvitz A, Kleinman S. Secretary or SecretarAI: Assessing the Triage Performance of AI Versus Human Staff in a Specialized Dental Clinic. Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology 2025 View
  43. Brown H. Evaluation of Artificial Intelligence for Patient Self-Triage: Comparison of General-Purpose AI Platforms With the NHS 111 Online Symptom Checker in the United Kingdom. Cureus 2025 View
  44. Wong A, Roberts M, Pantangco M, Arnold A, Philp A, Šlapeta J, Livingstone S. When used for veterinary triage, artificial intelligence models recognise emergencies but are more likely than veterinary staff to flag non‐urgent cases as urgent. Veterinary Record 2025 View