This paper is in the following e-collection/theme issue:

Generative Language Models Including ChatGPT (467) Artificial Intelligence (1676) Chatbots and Conversational Agents (617) Natural Language Processing (790)

Published on in Vol 26 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/48996, first published .
Automated Paper Screening for Clinical Reviews Using Large Language Models: Data Analysis Study

Automated Paper Screening for Clinical Reviews Using Large Language Models: Data Analysis Study

Automated Paper Screening for Clinical Reviews Using Large Language Models: Data Analysis Study

Authors of this article:

Eddie Guo1 Author Orcid Image ;   Mehul Gupta1 Author Orcid Image ;   Jiawen Deng2 Author Orcid Image ;   Ye-Jean Park2 Author Orcid Image ;   Michael Paget1 Author Orcid Image ;   Christopher Naugler1 Author Orcid Image
Article Authors Cited by (44) Tweetations (58) Metrics Author Video

Journals

  1. Kohandel Gargari O, Mahmoudi M, Hajisafarali M, Samiee R. Enhancing title and abstract screening for systematic reviews with GPT-3.5 turbo. BMJ Evidence-Based Medicine 2023:bmjebm-2023-112678 View
  2. Ye A, Maiti A, Schmidt M, Pedersen S. A Hybrid Semi-Automated Workflow for Systematic and Literature Review Processes with Large Language Model Analysis. Future Internet 2024;16(5):167 View
  3. Tran V, Gartlehner G, Yaacoub S, Boutron I, Schwingshackl L, Stadelmaier J, Sommer I, Alebouyeh F, Afach S, Meerpohl J, Ravaud P. Sensitivity and Specificity of Using GPT-3.5 Turbo Models for Title and Abstract Screening in Systematic Reviews and Meta-analyses. Annals of Internal Medicine 2024;177(6):791 View
  4. Yoon D, Han C, Kim D, Kim S, Bae S, Ryu J, Choi Y. Redefining Health Care Data Interoperability: Empirical Exploration of Large Language Models in Information Exchange. Journal of Medical Internet Research 2024;26:e56614 View
  5. Riaz I, Naqvi S, Hasan B, Murad M. Future of Evidence Synthesis: Automated, Living, and Interactive Systematic Reviews and Meta-analyses. Mayo Clinic Proceedings: Digital Health 2024;2(3):361 View
  6. Landschaft A, Antweiler D, Mackay S, Kugler S, Rüping S, Wrobel S, Höres T, Allende-Cid H. Implementation and evaluation of an additional GPT-4-based reviewer in PRISMA-based medical systematic literature reviews. International Journal of Medical Informatics 2024;189:105531 View
  7. Luo X, Chen F, Zhu D, Wang L, Wang Z, Liu H, Lyu M, Wang Y, Wang Q, Chen Y. Potential Roles of Large Language Models in the Production of Systematic Reviews and Meta-Analyses. Journal of Medical Internet Research 2024;26:e56780 View
  8. Menold H, Wieland V, Haney C, Uysal D, Wessels F, Cacciamani G, Michel M, Seide S, Kowalewski K. Machine learning enables automated screening for systematic reviews and meta-analysis in urology. World Journal of Urology 2024;42(1) View
  9. Akinseloyin O, Jiang X, Palade V. A question-answering framework for automated abstract screening using large language models. Journal of the American Medical Informatics Association 2024;31(9):1939 View
  10. Guo E, Ramchandani R, Park Y, Gupta M. OSCEai: personalized interactive learning for undergraduate medical education. Canadian Medical Education Journal 2024 View
  11. Park K, Choi H. How to Harness the Power of GPT for Scientific Research: A Comprehensive Review of Methodologies, Applications, and Ethical Considerations. Nuclear Medicine and Molecular Imaging 2024;58(6):323 View
  12. Matsui K, Utsumi T, Aoki Y, Maruki T, Takeshima M, Takaesu Y. Human-Comparable Sensitivity of Large Language Models in Identifying Eligible Studies Through Title and Abstract Screening: 3-Layer Strategy Using GPT-3.5 and GPT-4 for Systematic Reviews. Journal of Medical Internet Research 2024;26:e52758 View
  13. Li Y, Luan Z, Liu Y, Liu H, Qi J, Han D. Automated information extraction model enhancing traditional Chinese medicine RCT evidence extraction (Evi-BERT): algorithm development and validation. Frontiers in Artificial Intelligence 2024;7 View
  14. Klang E, Alper L, Sorin V, Barash Y, Nadkarni G, Zimlichman E. Advancing radiology practice and research: harnessing the potential of large language models amidst imperfections. BJR|Open 2023;6(1) View
  15. Arfaie S, Sadegh Mashayekhi M, Mofatteh M, Ma C, Ruan R, MacLean M, Far R, Saini J, Harmsen I, Duda T, Gomez A, Rebchuk A, Pingbei Wang A, Rasiah N, Guo E, Fazlollahi A, Rose Swan E, Amin P, Mohammed S, Atkinson J, Del Maestro R, Girgis F, Kumar A, Das S. ChatGPT and neurosurgical education: A crossroads of innovation and opportunity. Journal of Clinical Neuroscience 2024;129:110815 View
  16. Reason T, Langham J, Gimblett A. Automated Mass Extraction of Over 680,000 PICOs from Clinical Study Abstracts Using Generative AI: A Proof-of-Concept Study. Pharmaceutical Medicine 2024;38(5):365 View
  17. Lee J, Park S, Shin J, Cho B. Analyzing evaluation methods for large language models in the medical field: a scoping review. BMC Medical Informatics and Decision Making 2024;24(1) View
  18. Sugiura A, Saegusa S, Jin Y, Yoshimoto R, Smith N, Dohi K, Higuchi T, Kozu T. Evaluation of RMES, an Automated Software Tool Utilizing AI, for Literature Screening with Reference to Published Systematic Reviews as Case-Studies: Development and Usability Study. JMIR Formative Research 2024;8:e55827 View
  19. Bailey R, MacFarlane A, Field M, Tagkopoulos I, Baranzini S, Edwards K, Rose C, Schork N, Singhal A, Wallace B, Fisher K, Markakis K, Stover P, Bovell-Benjamin A. Artificial intelligence in food and nutrition evidence: The challenges and opportunities. PNAS Nexus 2024;3(12) View
  20. Liu Z, Chai Y, Li J. Toward Automated Simulation Research Workflow through LLM Prompt Engineering Design. Journal of Chemical Information and Modeling 2025;65(1):114 View
  21. Janumpally R, Nanua S, Ngo A, Youens K. Generative artificial intelligence in graduate medical education. Frontiers in Medicine 2025;11 View
  22. Hua Y, Beam A, Chibnik L, Torous J. From statistics to deep learning: Using large language models in psychiatric research. International Journal of Methods in Psychiatric Research 2025;34(1) View
  23. Schrager S, Seehusen D, Sexton S, Richardson C, Neher J, Pimlott N, Bowman M, Rodíguez J, Morley C, Li L, Dera J. Use of AI in family medicine publications: a joint editorial from journal editors. Evidence-Based Practice 2025;28(1):1 View
  24. Chen H, Jiang Z, Liu X, Xue C, Yew S, Sheng B, Zheng Y, Wang X, Wu Y, Sivaprasad S, Wong T, Chaudhary V, Tham Y. Can large language models fully automate or partially assist paper selection in systematic reviews?. British Journal of Ophthalmology 2025:bjo-2024-326254 View
  25. Fleurence R, Bian J, Wang X, Xu H, Dawoud D, Higashi M, Chhatwal J. Generative Artificial Intelligence for Health Technology Assessment: Opportunities, Challenges, and Policy Considerations: An ISPOR Working Group Report. Value in Health 2025;28(2):175 View
  26. Schrager S, Seehusen D, Sexton S, Richardson C, Neher J, Pimlott N, Bowman M, Rodríguez J, Morley C, Li L, DomDera J. Use of AI in family medicine publications: a joint editorial from journal editors. Family Medicine and Community Health 2025;13(1):e003238 View
  27. Rafiq K, Beery S, Palmer M, Harchaoui Z, Abrahms B. Generative AI as a tool to accelerate the field of ecology. Nature Ecology & Evolution 2025;9(3):378 View
  28. Purewal A, Fautsch K, Klasova J, Hussain N, D'Souza R. Human versus artificial intelligence: evaluating ChatGPT’s performance in conducting published systematic reviews with meta-analysis in chronic pain research. Regional Anesthesia & Pain Medicine 2025:rapm-2024-106358 View
  29. Cao C, Sang J, Arora R, Chen D, Kloosterman R, Cecere M, Gorla J, Saleh R, Drennan I, Teja B, Fehlings M, Ronksley P, Leung A, Weisz D, Ware H, Whelan M, Emerson D, Arora R, Bobrovitz N. Development of Prompt Templates for Large Language Model–Driven Screening in Systematic Reviews. Annals of Internal Medicine 2025;178(3):389 View
  30. Li Y, Datta S, Rastegar-Mojarad M, Lee K, Paek H, Glasgow J, Liston C, He L, Wang X, Xu Y. Enhancing systematic literature reviews with generative artificial intelligence: development, applications, and performance evaluation. Journal of the American Medical Informatics Association 2025;32(4):616 View
  31. Çalışkan E. Exploring possibilities and limits of ChatGPT: Usage in building design studies. Turkish Journal of Engineering 2025;9(3):490 View
  32. Colangelo M, Guizzardi S, Meleti M, Calciolari E, Galli C. How to Write Effective Prompts for Screening Biomedical Literature Using Large Language Models. BioMedInformatics 2025;5(1):15 View
  33. IIZUMI T, ONO Y, TAKIMOTO T, Chaogejilatu . Crop phenology data extraction from research papers using a large language model. Journal of Agricultural Meteorology 2025;81(2):112 View
  34. Sujau M, Wada M, Vallée E, Hillis N, Sušnjak T. Accelerating Disease Model Parameter Extraction: An LLM-Based Ranking Approach to Select Initial Studies for Literature Review Automation. Machine Learning and Knowledge Extraction 2025;7(2):28 View
  35. Brodsky V, Ullah E, Bychkov A, Song A, Walk E, Louis P, Rasool G, Singh R, Mahmood F, Bui M, Parwani A. Generative Artificial Intelligence in Anatomic Pathology. Archives of Pathology & Laboratory Medicine 2025;149(4):298 View
  36. Dietrich E. Artificial intelligence in key pricing, reimbursement, and market access (PRMA) processes: better, faster, cheaper—can you really pick two?. Journal of Medical Economics 2025;28(1):586 View
  37. Nitturi V, Flores A, Bauer D. Using Natural Language Processing to Automate Screening of Abstracts for Neurosurgical Guideline Creation. Neurosurgery 2025 View
  38. Boyle A, Huo B, Sylla P, Calabrese E, Kumar S, Slater B, Walsh D, Vosburg R. Large language model-generated clinical practice guideline for appendicitis. Surgical Endoscopy 2025 View
  39. Nykvist B, Macura B, Xylia M, Olsson E. Testing the utility of GPT for title and abstract screening in environmental systematic evidence synthesis. Environmental Evidence 2025;14(1) View

Conference Proceedings

  1. Huotala A, Kuutila M, Ralph P, Mäntylä M. Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering. The Promise and Challenges of Using LLMs to Accelerate the Screening Process of Systematic Reviews View
  2. Felizardo K, Lima M, Deizepe A, Conte T, Steinmacher I. Proceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. ChatGPT application in Systematic Literature Reviews in Software Engineering: an evaluation of its accuracy to support the selection activity View
  3. Sandner E, Hu B, Simiceanu A, Fontana L, Jakovljevic I, Henriques A, Wagner A, Gütl C. 2024 2nd International Conference on Foundation and Large Language Models (FLLM). Screening Automation for Systematic Reviews: A 5-Tier Prompting Approach Meeting Cochrane’s Sensitivity Requirement View
  4. Rahman M, Al-Hazzaa S. GLOBECOM 2024 - 2024 IEEE Global Communications Conference. Next-Generation Virtual Hospital: Integrating Discriminative and Large Multi-Modal Generative AI for Personalized Healthcare View
  5. Ogdu C, Gurbuz S, Karakose M, Hanoglu E. 2025 29th International Conference on Information Technology (IT). Medical Implications of LLM Based Clinical Decision Support Systems in Healthcare View