This paper is in the following e-collection/theme issue:

Clinical Information and Decision Making (1423) Machine Learning (1488)

Published on in Vol 22, No 11 (2020): November

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/22421, first published .
Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study

Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study

Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study

Authors of this article:

Sahil Sandhu1 Author Orcid Image ;   Anthony L Lin2 Author Orcid Image ;   Nathan Brajer2 Author Orcid Image ;   Jessica Sperling3 Author Orcid Image ;   William Ratliff4 Author Orcid Image ;   Armando D Bedoya5 Author Orcid Image ;   Suresh Balu4 Author Orcid Image ;   Cara O'Brien6 Author Orcid Image ;   Mark P Sendak4 Author Orcid Image
Article Authors Cited by (71) Tweetations (13) Metrics

Journals

  1. Castaldo R, Cavaliere C, Soricelli A, Salvatore M, Pecchia L, Franzese M. Radiomic and Genomic Machine Learning Method Performance for Prostate Cancer Diagnosis: Systematic Literature Review. Journal of Medical Internet Research 2021;23(4):e22394 View
  2. Zhong T, Zhuang Z, Dong X, Wong K, Wong W, Wang J, He D, Liu S. Predicting Antituberculosis Drug–Induced Liver Injury Using an Interpretable Machine Learning Method: Model Development and Validation Study. JMIR Medical Informatics 2021;9(7):e29226 View
  3. Hwang G, Tang K, Tu Y. How artificial intelligence (AI) supports nursing education: profiling the roles, applications, and trends of AI in nursing education research (1993–2020). Interactive Learning Environments 2024;32(1):373 View
  4. Hogg H, Al-Zubaidy M, Talks J, Denniston A, Kelly C, Malawana J, Papoutsi C, Teare M, Keane P, Beyer F, Maniatopoulos G. Stakeholder Perspectives of Clinical Artificial Intelligence Implementation: Systematic Review of Qualitative Evidence. Journal of Medical Internet Research 2023;25:e39742 View
  5. Sendak M, Gao M, Ratliff W, Nichols M, Bedoya A, O’Brien C, Balu S. Looking for clinician involvement under the wrong lamp post: The need for collaboration measures. Journal of the American Medical Informatics Association 2021;28(11):2541 View
  6. Sax D, Sturmer L, Mark D, Rana J, Reed M. Barriers and Opportunities Regarding Implementation of a Machine Learning-Based Acute Heart Failure Risk Stratification Tool in the Emergency Department. Diagnostics 2022;12(10):2463 View
  7. Taribagil P, Hogg H, Balaskas K, Keane P. Integrating artificial intelligence into an ophthalmologist’s workflow: obstacles and opportunities. Expert Review of Ophthalmology 2023;18(1):45 View
  8. Gonem S, Taylor A, Figueredo G, Forster S, Quinlan P, Garibaldi J, McKeever T, Shaw D. Dynamic early warning scores for predicting clinical deterioration in patients with respiratory disease. Respiratory Research 2022;23(1) View
  9. Pumplun L, Fecho M, Wahl N, Peters F, Buxmann P. Adoption of Machine Learning Systems for Medical Diagnostics in Clinics: Qualitative Interview Study. Journal of Medical Internet Research 2021;23(10):e29301 View
  10. von Gerich H, Moen H, Block L, Chu C, DeForest H, Hobensack M, Michalowski M, Mitchell J, Nibber R, Olalia M, Pruinelli L, Ronquillo C, Topaz M, Peltonen L. Artificial Intelligence -based technologies in nursing: A scoping literature review of the evidence. International Journal of Nursing Studies 2022;127:104153 View
  11. Ratzki-Leewing A, Ryan B, Zou G, Webster-Bogaert S, Black J, Stirling K, Timcevska K, Khan N, Buchenberger J, Harris S. Predicting Real-world Hypoglycemia Risk in American Adults With Type 1 or 2 Diabetes Mellitus Prescribed Insulin and/or Secretagogues: Protocol for a Prospective, 12-Wave Internet-Based Panel Survey With Email Support (the iNPHORM [Investigating Novel Predictions of Hypoglycemia Occurrence Using Real-world Models] Study). JMIR Research Protocols 2022;11(2):e33726 View
  12. Silvestri J, Kmiec T, Bishop N, Regli S, Weissman G. Desired Characteristics of a Clinical Decision Support System for Early Sepsis Recognition: Interview Study Among Hospital-Based Clinicians. JMIR Human Factors 2022;9(4):e36976 View
  13. Singer S, Kellogg K, Galper A, Viola D. Enhancing the value to users of machine learning-based clinical decision support tools: A framework for iterative, collaborative development and implementation. Health Care Management Review 2022;47(2):E21 View
  14. Kashyap S, Morse K, Patel B, Shah N. A survey of extant organizational and computational setups for deploying predictive models in health systems. Journal of the American Medical Informatics Association 2021;28(11):2445 View
  15. Ed-Driouch C, Mars F, Gourraud P, Dumas C. Addressing the Challenges and Barriers to the Integration of Machine Learning into Clinical Practice: An Innovative Method to Hybrid Human–Machine Intelligence. Sensors 2022;22(21):8313 View
  16. She W, Ang C, Neimeyer R, Burke L, Zhang Y, Jatowt A, Kawai Y, Hu J, Rauterberg M, Prigerson H, Siriaraya P. Investigation of a Web-Based Explainable AI Screening for Prolonged Grief Disorder. IEEE Access 2022;10:41164 View
  17. Jeong H, Kamaleswaran R. Pivotal challenges in artificial intelligence and machine learning applications for neonatal care. Seminars in Fetal and Neonatal Medicine 2022;27(5):101393 View
  18. Pan Y, Froese F. An interdisciplinary review of AI and HRM: Challenges and future directions. Human Resource Management Review 2023;33(1):100924 View
  19. Matthiesen S, Diederichsen S, Hansen M, Villumsen C, Lassen M, Jacobsen P, Risum N, Winkel B, Philbert B, Svendsen J, Andersen T. Clinician Preimplementation Perspectives of a Decision-Support Tool for the Prediction of Cardiac Arrhythmia Based on Machine Learning: Near-Live Feasibility and Qualitative Study. JMIR Human Factors 2021;8(4):e26964 View
  20. Schwartz J, George M, Rossetti S, Dykes P, Minshall S, Lucas E, Cato K. Factors Influencing Clinician Trust in Predictive Clinical Decision Support Systems for In-Hospital Deterioration: Qualitative Descriptive Study. JMIR Human Factors 2022;9(2):e33960 View
  21. Yang Z, Silcox C, Sendak M, Rose S, Rehkopf D, Phillips R, Peterson L, Marino M, Maier J, Lin S, Liaw W, Kakadiaris I, Heintzman J, Chu I, Bazemore A. Advancing primary care with Artificial Intelligence and Machine Learning. Healthcare 2022;10(1):100594 View
  22. Morris A, Horvat C, Stagg B, Grainger D, Lanspa M, Orme J, Clemmer T, Weaver L, Thomas F, Grissom C, Hirshberg E, East T, Wallace C, Young M, Sittig D, Suchyta M, Pearl J, Pesenti A, Bombino M, Beck E, Sward K, Weir C, Phansalkar S, Bernard G, Thompson B, Brower R, Truwit J, Steingrub J, Hiten R, Willson D, Zimmerman J, Nadkarni V, Randolph A, Curley M, Newth C, Lacroix J, Agus M, Lee K, deBoisblanc B, Moore F, Evans R, Sorenson D, Wong A, Boland M, Dere W, Crandall A, Facelli J, Huff S, Haug P, Pielmeier U, Rees S, Karbing D, Andreassen S, Fan E, Goldring R, Berger K, Oppenheimer B, Ely E, Pickering B, Schoenfeld D, Tocino I, Gonnering R, Pronovost P, Savitz L, Dreyfuss D, Slutsky A, Crapo J, Pinsky M, James B, Berwick D. Computer clinical decision support that automates personalized clinical care: a challenging but needed healthcare delivery strategy. Journal of the American Medical Informatics Association 2022;30(1):178 View
  23. Allen M, James C, Frost J, Liabo K, Pearn K, Monks T, Zhelev Z, Logan S, Everson R, James M, Stein K. Using simulation and machine learning to maximise the benefit of intravenous thrombolysis in acute stroke in England and Wales: the SAMueL modelling and qualitative study. Health and Social Care Delivery Research 2022;10(31):1 View
  24. Tulk Jesso S, Kelliher A, Sanghavi H, Martin T, Henrickson Parker S. Inclusion of Clinicians in the Development and Evaluation of Clinical Artificial Intelligence Tools: A Systematic Literature Review. Frontiers in Psychology 2022;13 View
  25. Park J, Hsu T, Hu J, Chen C, Hsu W, Lee M, Ho J, Lee C. Predicting Sepsis Mortality in a Population-Based National Database: Machine Learning Approach. Journal of Medical Internet Research 2022;24(4):e29982 View
  26. Hwang G, Chang P, Tseng W, Chou C, Wu C, Tu Y. Research Trends in Artificial Intelligence-Associated Nursing Activities Based on a Review of Academic Studies Published From 2001 to 2020. CIN: Computers, Informatics, Nursing 2022;40(12):814 View
  27. Abdel-Hafez A, Scott I, Falconer N, Canaris S, Bonilla O, Marxen S, Van Garderen A, Barras M. Predicting Therapeutic Response to Unfractionated Heparin Therapy: Machine Learning Approach. Interactive Journal of Medical Research 2022;11(2):e34533 View
  28. Zając H, Li D, Dai X, Carlsen J, Kensing F, Andersen T. Clinician-Facing AI in the Wild: Taking Stock of the Sociotechnical Challenges and Opportunities for HCI. ACM Transactions on Computer-Human Interaction 2023;30(2):1 View
  29. Chan S, Lee J, Ong M, Siddiqui F, Graves N, Ho A, Liu N. Implementation of Prediction Models in the Emergency Department from an Implementation Science Perspective—Determinants, Outcomes, and Real-World Impact: A Scoping Review. Annals of Emergency Medicine 2023;82(1):22 View
  30. Sandhu S, Sendak M, Ratliff W, Knechtle W, Fulkerson W, Balu S. Accelerating health system innovation: principles and practices from the Duke Institute for Health Innovation. Patterns 2023;4(4):100710 View
  31. van der Vegt A, Scott I, Dermawan K, Schnetler R, Kalke V, Lane P. Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework. Journal of the American Medical Informatics Association 2023;30(7):1349 View
  32. Lambert S, Madi M, Sopka S, Lenes A, Stange H, Buszello C, Stephan A. An integrative review on the acceptance of artificial intelligence among healthcare professionals in hospitals. npj Digital Medicine 2023;6(1) View
  33. Martinez-Ortigosa A, Martinez-Granados A, Gil-Hernández E, Rodriguez-Arrastia M, Ropero-Padilla C, Roman P, Leal-Costa C. Applications of Artificial Intelligence in Nursing Care: A Systematic Review. Journal of Nursing Management 2023;2023:1 View
  34. Gokhale S, Taylor D, Gill J, Hu Y, Zeps N, Lequertier V, Prado L, Teede H, Enticott J. Hospital length of stay prediction tools for all hospital admissions and general medicine populations: systematic review and meta-analysis. Frontiers in Medicine 2023;10 View
  35. Vo V, Chen G, Aquino Y, Carter S, Do Q, Woode M. Multi-stakeholder preferences for the use of artificial intelligence in healthcare: A systematic review and thematic analysis. Social Science & Medicine 2023;338:116357 View
  36. Wan Y, Wright M, McFarland M, Dishman D, Nies M, Rush A, Madaras-Kelly K, Jeppesen A, Del Fiol G. Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review. Journal of the American Medical Informatics Association 2023;31(1):256 View
  37. Kim J, Ryan K, Kasun M, Hogg J, Dunn L, Roberts L. Physicians’ and Machine Learning Researchers’ Perspectives on Ethical Issues in the Early Development of Clinical Machine Learning Tools: Qualitative Interview Study. JMIR AI 2023;2:e47449 View
  38. Wang S, Hogg H, Sangvai D, Patel M, Weissler E, Kellogg K, Ratliff W, Balu S, Sendak M. Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study. JMIR Formative Research 2023;7:e43963 View
  39. Aslan A, Permana B, Harris P, Naidoo K, Pienaar M, Irwin A. The Opportunities and Challenges for Artificial Intelligence to Improve Sepsis Outcomes in the Paediatric Intensive Care Unit. Current Infectious Disease Reports 2023;25(11):243 View
  40. Choi A, Choi S, Chung K, Chung H, Song T, Choi B, Kim J. Development of a machine learning-based clinical decision support system to predict clinical deterioration in patients visiting the emergency department. Scientific Reports 2023;13(1) View
  41. Bhat J, Feng X, Mir Z, Raina A, Siddique K. Recent advances in artificial intelligence, mechanistic models, and speed breeding offer exciting opportunities for precise and accelerated genomics‐assisted breeding. Physiologia Plantarum 2023;175(4) View
  42. Verma A, Trbovich P, Mamdani M, Shojania K. Grand rounds in methodology: key considerations for implementing machine learning solutions in quality improvement initiatives. BMJ Quality & Safety 2024;33(2):121 View
  43. Casey S, Reed M, LeMaster C, Mark D, Gaskin J, Norris R, Sax D. Physicians’ Perceptions of Clinical Decision Support to Treat Patients With Heart Failure in the ED. JAMA Network Open 2023;6(11):e2344393 View
  44. Lu G, Zhang J, Shi T, Liu Y, Gao X, Zeng Q, Ding J, Chen J, Yang K, Ma Q, Liu X, Ren C, Yu H, Li Y, Ponraj V. Development and application of a nomogram model for the prediction of carbapenem-resistant Klebsiella pneumoniae infection in neuro-ICU patients. Microbiology Spectrum 2024;12(1) View
  45. Kasun M, Ryan K, Paik J, Lane-McKinley K, Dunn L, Roberts L, Kim J. Academic machine learning researchers’ ethical perspectives on algorithm development for health care: a qualitative study. Journal of the American Medical Informatics Association 2024;31(3):563 View
  46. Barwise A, Curtis S, Diedrich D, Pickering B. Using artificial intelligence to promote equitable care for inpatients with language barriers and complex medical needs: clinical stakeholder perspectives. Journal of the American Medical Informatics Association 2024;31(3):611 View
  47. Ueda D, Walston S, Matsumoto T, Deguchi R, Tatekawa H, Miki Y. Evaluating GPT-4-based ChatGPT's clinical potential on the NEJM quiz. BMC Digital Health 2024;2(1) View
  48. O'Connor S, Vercell A, Wong D, Yorke J, Fallatah F, Cave L, Anny Chen L. The application and use of artificial intelligence in cancer nursing: A systematic review. European Journal of Oncology Nursing 2024;68:102510 View
  49. Evans R, Bryant L, Russell G, Absolom K. Trust and acceptability of data-driven clinical recommendations in everyday practice: A scoping review. International Journal of Medical Informatics 2024;183:105342 View
  50. Giddings R, Joseph A, Callender T, Janes S, van der Schaar M, Sheringham J, Navani N. Factors influencing clinician and patient interaction with machine learning-based risk prediction models: a systematic review. The Lancet Digital Health 2024;6(2):e131 View
  51. Rony M, Kayesh I, Bala S, Akter F, Parvin M. Artificial intelligence in future nursing care: Exploring perspectives of nursing professionals - A descriptive qualitative study. Heliyon 2024;10(4):e25718 View
  52. Salybekov A, Wolfien M, Hahn W, Hidaka S, Kobayashi S. Artificial Intelligence Reporting Guidelines’ Adherence in Nephrology for Improved Research and Clinical Outcomes. Biomedicines 2024;12(3):606 View
  53. Palmowski L, Nowak H, Witowski A, Koos B, Wolf A, Weber M, Kleefisch D, Unterberg M, Haberl H, von Busch A, Ertmer C, Zarbock A, Bode C, Putensen C, Limper U, Wappler F, Köhler T, Henzler D, Oswald D, Ellger B, Ehrentraut S, Bergmann L, Rump K, Ziehe D, Babel N, Sitek B, Marcus K, Frey U, Thoral P, Adamzik M, Eisenacher M, Rahmel T, Lazzeri C. Assessing SOFA score trajectories in sepsis using machine learning: A pragmatic approach to improve the accuracy of mortality prediction. PLOS ONE 2024;19(3):e0300739 View
  54. Cheng R, Aggarwal A, Chakraborty A, Harish V, McGowan M, Roy A, Szulewski A, Nolan B. Implementation considerations for the adoption of artificial intelligence in the emergency department. The American Journal of Emergency Medicine 2024;82:75 View
  55. Secor A, Justafort J, Torrilus C, Honoré J, Kiche S, Sandifer T, Beima-Sofie K, Wagner A, Pintye J, Puttkammer N. “Following the data”: Perceptions of and willingness to use clinical decision support tools to inform HIV care among Haitian clinicians. Health Policy and Technology 2024;13(3):100880 View
  56. Peterson K, Chapman A, Widanagamaachchi W, Sutton J, Ochoa B, Jones B, Stevens V, Classen D, Jones M, Liang H. Automating detection of diagnostic error of infectious diseases using machine learning. PLOS Digital Health 2024;3(6):e0000528 View
  57. Wieben A, Alreshidi B, Douthit B, Sileo M, Vyas P, Steege L, Gilmore‐Bykovskyi A. Nurses' perceptions of the design, implementation, and adoption of machine learning clinical decision support: A descriptive qualitative study. Journal of Nursing Scholarship 2024 View
  58. Strechen I, Wilson P, Eltalhi T, Piche K, Tschida-Reuter D, Howard D, Sutor B, Tiong I, Herasevich S, Pickering B, Barwise A. Harnessing health information technology to promote equitable care for patients with limited English proficiency and complex care needs. Trials 2024;25(1) View
  59. Lazzarino R, Borek A, Honeyford K, Welch J, Brent A, Kinderlerer A, Cooke G, Patil S, Gordon A, Glampson B, Goodman P, Ghazal P, Daniels R, Costelloe C, Tonkin-Crine S. Views and Uses of Sepsis Digital Alerts in National Health Service Trusts in England: Qualitative Study With Health Care Professionals. JMIR Human Factors 2024;11:e56949 View
  60. Griffin A, Wang K, Leung T, Facelli J. Recommendations to promote fairness and inclusion in biomedical AI research and clinical use. Journal of Biomedical Informatics 2024;157:104693 View
  61. Ayorinde A, Mensah D, Walsh J, Ghosh I, Ibrahim S, Hogg J, Peek N, Griffiths F. Health Care Professionals’ Experience of Using AI: Systematic Review With Narrative Synthesis. Journal of Medical Internet Research 2024;26:e55766 View
  62. Kamel Rahimi A, Pienaar O, Ghadimi M, Canfell O, Pole J, Shrapnel S, van der Vegt A, Sullivan C. Implementing AI in Hospitals to Achieve a Learning Health System: Systematic Review of Current Enablers and Barriers. Journal of Medical Internet Research 2024;26:e49655 View
  63. Ramgopal S, Macy M, Hayes A, Florin T, Carroll M, Kshetrapal A. Clinician Perspectives on Decision Support and AI-based Decision Support in a Pediatric ED. Hospital Pediatrics 2024;14(10):828 View
  64. Maniscalco M, Candia C, Fuschillo S, Ambrosino P, Paris D, Motta A. Exhaled breath condensate (EBC) in respiratory diseases: recent advances and future perspectives in the age of omic sciences. Journal of Breath Research 2024;18(4):045001 View
  65. Owoyemi A, Okpara E, Salwei M, Boyd A. End user experience of a widely used artificial intelligence based sepsis system. JAMIA Open 2024;7(4) View
  66. Avci C, Bagca B, Shademan B, Takanlou L, Takanlou M, Nourazarian A. Machine learning in oncological pharmacogenomics: advancing personalized chemotherapy. Functional & Integrative Genomics 2024;24(5) View
  67. Su M, Wang D. Intelligent Reconstruction and Effectiveness Evaluation of Nursing Workflow for Future Hospitals. Applied Mathematics and Nonlinear Sciences 2024;9(1) View
  68. Preti L, Ardito V, Compagni A, Petracca F, Cappellaro G. Implementation of Machine Learning Applications in Health Care Organizations: Systematic Review of Empirical Studies. Journal of Medical Internet Research 2024;26:e55897 View

Books/Policy Documents

  1. Lim D, Loh B, Vong W, Then P. Deep Learning Theory and Applications. View