This paper is in the following e-collection/theme issue:

Artificial Intelligence (1675) Generative Language Models Including ChatGPT (467) Epublishing and Open Access (95) Reviews in Medical Education (69) New Methods and Approaches in Medical Education (301) Methods and Instruments in Medical Informatics (164) Chatbots and Conversational Agents (616)

Published on in Vol 26 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/53164, first published .
Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis

Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis

Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis

Authors of this article:

Mikaël Chelli1 Author Orcid Image ;   Jules Descamps2 Author Orcid Image ;   Vincent Lavoué1 Author Orcid Image ;   Christophe Trojani1 Author Orcid Image ;   Michel Azar1 Author Orcid Image ;   Marcel Deckert3 Author Orcid Image ;   Jean-Luc Raynier1 Author Orcid Image ;   Gilles Clowez1 Author Orcid Image ;   Pascal Boileau1 Author Orcid Image ;   Caroline Ruetsch-Chelli3 Author Orcid Image
Article Authors Cited by (36) Tweetations (10) Metrics

Journals

  1. Ren X, Wei Q. Sligpt: A Large Language Model-Based Approach for Data Dependency Analysis on Solidity Smart Contracts. Software 2024;3(3):345 View
  2. Adam G, DeYoung J, Paul A, Saldanha I, Balk E, Trikalinos T, Wallace B. Literature search sandbox: a large language model that generates search queries for systematic reviews. JAMIA Open 2024;7(3) View
  3. Zhang D, Ma Z, Gong R, Lian L, Li Y, He Z, Han Y, Hui J, Huang J, Jiang J, Weng W, Feng J. Using Natural Language Processing (GPT-4) for Computed Tomography Image Analysis of Cerebral Hemorrhages in Radiology: Retrospective Analysis. Journal of Medical Internet Research 2024;26:e58741 View
  4. Mija D, Kehlet H, Rosero E, Joshi G. Evaluating the role of ChatGPT in perioperative pain management versus procedure-specific postoperative pain management (PROSPECT) recommendations. British Journal of Anaesthesia 2024;133(6):1318 View
  5. Sanduleanu S, Ersahin K, Bremm J, Talibova N, Damer T, Erdogan M, Kottlors J, Goertz L, Bruns C, Maintz D, Abdullayev N. Feasibility of GPT-3.5 versus Machine Learning for Automated Surgical Decision-Making Determination: A Multicenter Study on Suspected Appendicitis. AI 2024;5(4):1942 View
  6. Thimm H, Rasmussen K. ChatGPT discovery of green image damaging information for large production companies. Journal of Cleaner Production 2024;478:143978 View
  7. Yan L, Greiff S, Teuber Z, Gašević D. Promises and challenges of generative artificial intelligence for human learning. Nature Human Behaviour 2024;8(10):1839 View
  8. Ivanisenko T, Demenkov P, Ivanisenko V. An Accurate and Efficient Approach to Knowledge Extraction from Scientific Publications Using Structured Ontology Models, Graph Neural Networks, and Large Language Models. International Journal of Molecular Sciences 2024;25(21):11811 View
  9. Iglesias S, Earp B, Voinea C, Mann S, Zahiu A, Jecker N, Savulescu J. Digital Doppelgängers and Lifespan Extension: What Matters?. The American Journal of Bioethics 2025;25(2):95 View
  10. Tomczyk P, Brüggemann P, Mergner N, Petrescu M. Are AI tools better than traditional tools in literature searching? Evidence from E-commerce research. Journal of Librarianship and Information Science 2024 View
  11. Goto H, Shiraishi Y, Okada S. Performance Evaluation of GPT-4o and o1-Preview Using the Certification Examination for the Japanese 'Operations Chief of Radiography With X-rays'. Cureus 2024 View
  12. Gunesli I, Aksun S, Fathelbab J, Yildiz B. Comparative evaluation of ChatGPT-4, ChatGPT-3.5 and Google Gemini on PCOS assessment and management based on recommendations from the 2023 guideline. Endocrine 2024;88(1):315 View
  13. Atkinson C. AI-pocalypse now: Automating the systematic literature review with SPARK (Systematic processing and automated review Kit) – gathering, organising, filtering, and scaffolding.. MethodsX 2025;14:103129 View
  14. Cohen J, Moher D. Generative artificial intelligence and academic writing: friend or foe?. Journal of Clinical Epidemiology 2025;179:111646 View
  15. Gao C, Hu X, Gao S, Xia X, Jin Z. The Current Challenges of Software Engineering in the Era of Large Language Models. ACM Transactions on Software Engineering and Methodology 2025 View
  16. Zhang C, Zhao T, Saraoglu H, Louton D. A Benchmark Comparison of a Domain-Focused Pipeline with ChatGPT. Journal of Computer Information Systems 2025:1 View
  17. Jones N. AI hallucinations can’t be stopped — but these techniques can limit their damage. Nature 2025;637(8047):778 View
  18. Ratuszniak A, Gos E, Lorens A, Skarzynski P, Skarzynski H, Jedrzejczak W. Performance of ChatGPT in Pediatric Audiology as Rated by Students and Experts. Journal of Clinical Medicine 2025;14(3):875 View
  19. Cinquin O. Steering veridical large language model analyses by correcting and enriching generated database queries: first steps toward ChatGPT bioinformatics. Briefings in Bioinformatics 2024;26(1) View
  20. Hidalgo-Betanzos J, Prol-Godoy I, Terés-Zubiaga J, Briones-Llorente R, Martín-Garín A. Can ChatGPT AI Replace or Contribute to Experts’ Diagnosis for Renovation Measures Identification?. Buildings 2025;15(3):421 View
  21. Erdem O, Hassett K, Egriboyun F. Hallucination in AI-generated financial literature reviews: evaluating bibliographic accuracy. International Journal of Data Science and Analytics 2025 View
  22. Bracken A, Reilly C, Feeley A, Sheehan E, Merghani K, Feeley I. Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review. Journal of Medical Systems 2025;49(1) View
  23. Kiyomiya K, Aomori T, Ohtani H. Medication counseling for OTC drugs using customized ChatGPT-4: Comparison with ChatGPT-3.5 and ChatGPT-4o. DIGITAL HEALTH 2025;11 View
  24. Gupta A, Basha A, Sontam T, Hlavinka W, Croen B, Abdou C, Abdullah M, Hamilton R. Evolution of patient education materials from large-language artificial intelligence models on complex regional pain syndrome: are patients learning?. Baylor University Medical Center Proceedings 2025;38(3):221 View
  25. Lorenc-Kukula K. Cutting-edge AI tools revolutionizing scientific research in life sciences. BioTechnologia 2025 View
  26. Scheinkman R, Kraft G, Kasheri E, Nouri K. Determining if ChatGPT‐4o Simplification of Mohs Postoperative Instructions Affects Instruction Quality. International Journal of Dermatology 2025 View
  27. Jacob C, Kerrigan P, Bastos M. The chat-chamber effect: Trusting the AI hallucination. Big Data & Society 2025;12(1) View
  28. Šuto Pavičić J, Marušić A, Buljan I. Using ChatGPT to Improve the Presentation of Plain Language Summaries of Cochrane Systematic Reviews About Oncology Interventions: Cross-Sectional Study. JMIR Cancer 2025;11:e63347 View
  29. Kula B, Kula A, Bagcier F, Alyanak B. Artificial intelligence solutions for temporomandibular joint disorders: Contributions and future potential of ChatGPT. Korean Journal of Orthodontics 2025;55(2):131 View
  30. Velasco A, Coelho E, Paes V, Chaves R, Oliveira R. A Inteligência Artificial na saúde: os Chatbots e suas aplicações na educação e pesquisa científica médicas. Caderno Pedagógico 2025;22(5):e15048 View
  31. Mun I, Hwang K. Exploring the Influence of Prompt Self-Efficacy: Accurate and Customized Information, Perceived Ease of Use, Satisfaction, and Continuance Intention to Use ChatGPT. International Journal of Human–Computer Interaction 2025:1 View
  32. Jongkind R, Elings E, Joukes E, Broens T, Leopold H, Wiesman F, Meinema J. Is your curriculum GenAI-proof? A method for GenAI impact assessment and a case study. MedEdPublish 2025;15:11 View
  33. Katz G, Zloto O, Hostovsky A, Huna-Baron R, Ben-Bassat Mizrachi I, Burgansky Z, Skaat A, Vishnevskia-Dai V, Fabian I, Sagiv O, Priel A, Glicksberg B, Klang E. Chat GPT vs an experienced ophthalmologist: evaluating chatbot writing performance in ophthalmology. Eye 2025 View
  34. Ostrovsky A. Evaluating a large language model's accuracy in chest X-ray interpretation for acute thoracic conditions. The American Journal of Emergency Medicine 2025;93:99 View
  35. Urban M, Brom C, Lukavský J, Děchtěrenko F, Hein V, Svacha F, Kmoníčková P, Urban K. “ChatGPT can make mistakes. Check important info.” Epistemic beliefs and metacognitive accuracy in students' integration of ChatGPT content into academic writing. British Journal of Educational Technology 2025 View
  36. Matsutomo N, Fukami M, Yamamoto T. Can interactive artificial intelligence be used for patient explanations of nuclear medicine examinations in Japanese?. Annals of Nuclear Medicine 2025 View