Background: Artificial intelligence (AI) is often heralded as a potential disruptor that will transform the practice of medicine. The amount of data collected and available in health care, coupled with advances in computational power, has contributed to advances in AI and an exponential growth of publications. However, the development of AI applications does not guarantee their adoption into routine practice. There is a risk that despite the resources invested, benefits for patients, staff, and society will not be realized if AI implementation is not better understood.
Objective: The aim of this study was to explore how the implementation of AI in health care practice has been described and researched in the literature by answering 3 questions: What are the characteristics of research on implementation of AI in practice? What types and applications of AI systems are described? What characteristics of the implementation process for AI systems are discernible?
Methods: A scoping review was conducted of MEDLINE (PubMed), Scopus, Web of Science, CINAHL, and PsycINFO databases to identify empirical studies of AI implementation in health care since 2011, in addition to snowball sampling of selected reference lists. Using Rayyan software, we screened titles and abstracts and selected full-text articles. Data from the included articles were charted and summarized.
Results: Of the 9218 records retrieved, 45 (0.49%) articles were included. The articles cover diverse clinical settings and disciplines; most (32/45, 71%) were published recently, were from high-income countries (33/45, 73%), and were intended for care providers (25/45, 56%). AI systems are predominantly intended for clinical care, particularly clinical care pertaining to patient-provider encounters. More than half (24/45, 53%) possess no action autonomy but rather support human decision-making. The focus of most research was on establishing the effectiveness of interventions (16/45, 35%) or related to technical and computational aspects of AI systems (11/45, 24%). Focus on the specifics of implementation processes does not yet seem to be a priority in research, and the use of frameworks to guide implementation is rare.
Conclusions: Our current empirical knowledge derives from implementations of AI systems with low action autonomy and approaches common to implementations of other types of information systems. To develop a specific and empirically based implementation framework, further research is needed on the more disruptive types of AI systems being implemented in routine care and on aspects unique to AI implementation in health care, such as building trust, addressing transparency issues, developing explainable and interpretable solutions, and addressing ethical concerns around privacy and data protection.
Artificial intelligence (AI) is often heralded as a potential disruptor that will transform the practice of medicine [, ]. The promise of AI lies in its ability to process and learn from large volumes of data and capture patterns otherwise difficult for humans to identify. This ability has raised questions and worries about liability and risks, in particular related to the level of autonomy granted to AI applications [ ]. Others see a role complementary to humans; for example, decision support or decision augmentation where humans (in the roles of clinicians or programmers) provide oversight and collaborate [ - ]. The latter approach has been demonstrated to yield superior performance compared with experts alone [ ]. Other benefits include improved patient outcomes, error reduction, health system optimization, cost reductions, and increased value [ ].
The amount of data collected and available in health care, coupled with advances in computational power, has contributed to advances in AI applications  and an exponential growth of publications on AI in health care, with >10,000 records on PubMed in 2021 alone. Included in this are multiple reviews across medical specialties that explore the potential roles of AI to augment health care delivery [ - ]. These include diagnostic (eg, early cancer diagnosis, diabetes retinopathy screening, or COVID-19 diagnosis based on computed tomography images), therapeutic (eg, precision medicine in chemotherapy and for combination drug therapy), and regulatory or administrative applications (eg, coding of records or economic evaluations), as well as for population health management (eg, public health surveillance or predictive epidemiological modeling) [ - ].
However, the development of AI applications does not guarantee their adoption into routine health care practice. Research has identified a number of factors influencing adoption of innovations. These include context (eg, economic and political context, laws and regulations, and sociocultural factors), organization (eg, organizational structure, resources, and processes), group (eg, professional values and cultures), individual (eg, attitudes, motivation, user satisfaction, and trust), and technology (eg, usability, design, accuracy, and explainability) [, ]. This suggests a need to know more about how AI can be implemented in health care, not only as an innovation but also with respect to its unique potential and associated concerns.
Previous reviews have tended to focus only on some aspects of the process of implementation of AI in health care; for example, regulation and legal issues [, ], trust and ethics [ - ], clinical and patient outcomes [ - ], and economic impact [ ]. Others have focused their studies on specific AI applications for health care, such as predictive medicine, diagnostics, and clinical decision-making [ , , , ]. A few reviews have been more overarching, focusing on coproduction processes [ ], implementation frameworks [ ], and critical implementation barriers or success factors [ ] that could inform the development of relevant implementation strategies of AI technology. Generally, it is argued that the implementation of AI in health care could significantly improve patient and health care outcomes, but none of these reviews have actually explored the knowledge base of real-world implementation in everyday clinical practice.
Given the resources invested in developing AI applications and the risk of reproducing already investigated aspects of effective AI applications to support, augment, and perhaps even transform health care for patients, staff, and society, we sought to explore how the implementation of AI in health care practice has been empirically investigated in the research literature.
We chose a scoping review methodology in line with the Arksey and O’Malley framework  and reported according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist ( ) [ ]. A previous review suggested that implementation of AI in health care was not well studied [ ]. A scoping review would thus enable a mapping of the “extent, range and nature of research activity” in this emerging area of research [ ].
Identifying the Research Question
To address our aim, we formulated three research questions:
- What are the characteristics of research on implementation of AI in practice?
- What types and applications of AI systems are described?
- What characteristics of the implementation process for AI systems are discernible?
Identifying Relevant Studies
We focused our search, with support from a university librarian, by iteratively testing synonyms for 3 concepts: artificial intelligence, health care, and implementation (). For the purposes of clarity, we differentiated between AI algorithms and models (the actual code), AI applications (the innovation package), and AI systems (the application in its context) and used standardized Medical Subject Headings terms and subject headings describing AI and its subcategories provided by the databases used for our searches [ ]. Implementation was defined as “An intentional effort designed to change or adapt or uptake interventions into routines,” based on a review of frameworks for the translation of AI into health care practice [ ]. Synonyms were joined by the Boolean operator OR; next, we combined the search strings for each concept with the Boolean operator AND ( ).
To cover content in both general and health- and health care–specific sources, 5 electronic databases were searched: MEDLINE (PubMed), Scopus, Web of Science, CINAHL, and PsycINFO. In addition, we used snowball sampling by manually reviewing reference lists of the review articles we had identified during the screening that might contain relevant references given the topic of the review.
Concept areas and synonyms used to develop the search strategy.
Search concepts, combined using “AND”
- Artificial Intelligence
Search terms, combined using “OR”
- Artificial intelligence, Neural networks, Deep learning, Machine learning
- Delivery of healthcare, Health care, Healthcare
- Implementation, Improvement, Innovation, Intervention
We included peer-reviewed empirical studies published in English between December 2011 and February 2022 because preliminary searches suggested that AI applications in health care are a more recent phenomenon ().
|Eligibility criteria and variable||Rationale|
|Peer reviewed||Greater credibility because the papers have been reviewed by peer experts in the field|
|Empirical study design||Empirical studies improve the ability to answer the research questions compared with conceptual commentaries or viewpoints|
|Published between December 2011 and February 2022||Given the rapid pace of development of technology and changing data sets, solutions developed before the last decade are likely to be obsolete|
|English language||Practical consideration, given the investigators’ language proficiency|
|Nonempirical designs, including editorials, commentaries, opinion articles, and reports||Empirical studies improve the ability to answer the research questions compared with conceptual commentaries or viewpoints|
|Proof-of-concept, feasibility, or validation studies not related to implementation of artificial intelligence technologies||As the aim was to explore implementation in practice, studies that stop short of that, for example, proof-of-concept, validity, or feasibility studies, should be excluded|
All identified records were imported into the open-access software Rayyan. Duplicates were removed, and the titles and abstracts of the remaining records were screened for eligibility by at least one of the authors. Any uncertainty or conflict was discussed at regular check-ins until consensus was reached among all authors. These discussions were informed by the multidisciplinary backgrounds of the authors. We also continually reviewed our interpretations of the screening criteria, and when questions were raised, we backtracked to ensure that the criteria had been applied correctly and in a universal fashion, independent of who had screened the records. We used the AI screening and highlighting function of Rayyan, but we still screened each record. We also erred on the side of inclusion. Full-text articles were then screened independently by at least two researchers. Conflicts and uncertainty were again resolved through discussion until consensus was reached among all researchers. As we followed the original framework, a quality appraisal of the included studies was not conducted.
Charting the Data
We developed a data extraction template to chart data for each of the research questions. To define these conceptual areas, we adopted the World Health Organization’s guidance on ethics and governance of AI for health definition of AI (based on a recommendation of the Council on Artificial Intelligence of the Organisation for Economic Co-operation and Development states) [, ]: “An AI system is a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments. AI systems are designed to operate with varying levels of autonomy” [ ].
The following data were extracted:
- General information: authors, publication year, country, clinical setting, study aim, and study design
- Types and applications of AI: AI technology used, type of AI model, type of task performed by AI, level of action autonomy, intended use of AI, and intended user of AI
- Implementation process: research focus, motives for implementation, elements in the implementation process, and frameworks used
Collating, Summarizing, and Reporting the Results
The extracted data relating to research questions 1 and 2 were mapped and summarized. A qualitative thematic analysis  was used to analyze data associated with research question 3 to summarize the motives for implementation and elements in the implementation process. Articles were read and reread, with initial ideas sorted into either the domain Motives behind the implementation or Elements in the implementation process. Next, initial codes were identified in each article. The codes were compared based on similarities and differences and collated into potential themes, which were then compared to generate a thematic map that was used to generate clear definitions and names for each theme in the respective domains. Coding and data analysis were performed in pairs, and any uncertainties were discussed among all authors until consensus was achieved.
We identified 9218 records, of which 9179 (99.58%) were identified through database searches and 39 (0.04%) through a snowball search of reference lists in the review articles (n=36). Of the 9218 records, after removal of duplicates, 5666 (61.47%) records remained, and we screened titles and abstracts. In this screening, 98% (5553/5666) of the records were excluded, and the remaining 2% (113/5666) were assessed for eligibility through full-text review. Of these 113 articles, 68 (60.2%) were excluded for reasons highlighted in, and 45 (39.8%) were included in the scoping review.
Research Question 1: Study Characteristics
The reviewed body of literature was fairly recent, with the majority of the studies (32/45, 71%) having been published between 2020 and 2022 [- ]. Most (33/45, 73%) of the articles were from North America and Europe [ , , - , , , - , - , - ], of which most (18/33, 55%) were from the United States [ , , - , , , - , - , , ]. The greatest number of AI systems were implemented either in hospital-wide settings (6/45, 13%) [ , , , , , ] or in radiology (6/45, 13%) [ , , , , , ]. Most (27/45, 60%) of the studies were authored by a multidisciplinary team [ , , - , , , , , , , , , , , , - , , - ], with clinical and IT or informatics backgrounds being the most common combination (9/27, 33%) [ , , , , , , , , ]. Among studies with authors from only 1 domain, the most common background was clinical (8/45, 18%) [ , , , , , , , ]. There was a wide range of study designs. Most (24/45, 53%) used a case-study design, including both single-case [ , , , , , - , , , - , , , , - , , ] or multiple comparative case designs [ , , ] ( and ).
|Author, year, country; clinical setting||Study aim||Study design|
|Anand et al , 2018, United States; pediatrics||Describe Child Health Improvement through Computer Automation system and methods to represent pediatric guidelines using Arden syntax||Case study|
|Baxter et al , 2020, United States; hospital-wide implementation||Conduct a detailed analysis of barriers to use of machine learning model in health care||Case study|
|Bennet , 2011, United States; mental health||Evaluate the effects of a data-driven clinical productivity system that leverages electronic health record data to provide productivity decision support functionality in a real-world clinical setting||Pre-post study|
|Champion et al , 2011, United States; intensive care||Illuminate barriers and facilitators to use of intensive insulin therapy CDSSa||Qualitative study|
|Chonde et al , 2021, United States; radiology||Evaluate the implementation of an AIb-powered translation system in radiology||Case study|
|Chong et al , 2021, Australia; hospital-wide implementation||Determine if a VTEc stewardship program can increase risk-appropriate VTE prophylaxis and VTE risk assessment using CDSS||Interrupted time series|
|Cruz et al , 2019, Spain; primary care||Describe a real-time CDSS and its effect on adherence to clinical pathways||Case study|
|Damoah et al , 2021, Ghana; management||Explore how an AI-enhanced medical drone application in Ghana’s health care supply chain improves the health care supply chain system||Case study|
|Davis et al , 2020, United States; radiology||Determine the impact of a machine learning algorithm, meant to mark CTd head examinations pending interpretation as higher probability for intracranial hemorrhage||Case study|
|Dios et al , 2015, Spain; surgery||Present a decision support system for operating room scheduling at a university hospital in Seville, Spain||Case study|
|García Bermúdez et al , 2021, Spain; internal medicine service||Assess the user satisfaction of a virtual caregiver designed to monitor the health of patients admitted to hospital for COVID-19 infection for a period of 30 days after discharge||Quantitative study|
|Goncalves et al , 2020, Brazil; nursing||Present the nurses’ experience with technological tools to support the early identification of sepsis||Case study|
|Herman et al , 2021, Indonesia; public health||Assess the impact of an AI-based application on rifampicin-resistant tuberculosis screening||Qualitative study with key informant interviews|
|Kalil et al , 2018, Brazil; surgery||Describe the impact of a new risk-management cognitive robot related to the processes of identification and care for patients at sepsis risk in a clinical-surgical unit||Retrospective observational study|
|Kashyap et al , 2021, United States; not specified||Identify the different computational and organizational setups that early-adopter health systems have used to integrate an AI-based CDSS into clinical workflows||Qualitative study with key informant interviews|
|Lacey et al , 2020, United Kingdom; surgery||Assess the impact of using automatic video auditing in the quality and quantity of hand-wash events||Interrupted time series|
|Lai et al , 2020, United States; public health||Describe the implementation of a digitally automated prehospital triage solution to direct patients to appropriate care||Case study|
|Litvin et al , 2012, United States; primary care||Describe use of a CDSS on antibiotic prescribing for acute respiratory infections in primary care, as well as facilitators and barriers to adoption||Mixed methods|
|McKillop et al , 2021, multiple regions; public health||Characterize the diverse use cases of COVID-19–related conversational agents built using the IBM Watson Assistant platform||Cross-sectional study|
|Mohamed et al , 2021, United Arab Emirates; dentistry||Validate and implement the AI system and quantify referral patterns to the orthodontist specialist before and after implementation of the system||Quantitative survey|
|Moorman , 2021, United States; inpatient care||Describe the experiences and lessons learned during implementation of AI system||Case study|
|Morales et al , 2021, Brazil; emergency care||Describe early implementation of a digital triage and monitoring service that included the use of a chatbot using algorithmic decision-making||Observational study|
|Ng et al , 2021, Singapore; general care||Develop a predictive model for risk stratification for enrollment into a nationwide transitional care program||Analysis of existing data set|
|O’Neil et al , 2021, United States; radiology||Assess (1) whether the introduction of an algorithm for the detection of intracerebral hemorrhage at noncontrast CT affects turnaround times and (2) whether the impact on turnaround time was dependent on the manner in which information was presented in the radiologist workflow||Quasi-experimental study|
|Petitgand et al , 2020, Canada; emergency department||Analyze the implementation of an AI-based decision support system in an emergency department focusing on actors’ representations of the system||Case study|
|Rais et al , 2018, Portugal; management||Discuss optimization approaches for logistics services in hospitals||Case study|
|Rath et al , 2017, United States; surgery||Describe the development, implementation, and evaluation of a model-based decision support system to determine daily scheduling of anesthesiologists and rooms for elective surgeries||Case study|
|Reis et al , 2020, Germany; hospital-wide implementation||Describe a failed AI project at a large hospital and identify the root causes that led to failure||Case study|
|Romero-Brufau et al , 2020, United States; primary care||To explore attitudes about AI among staff who used AI-based CDSS||Pre-post study|
|Romero-Brufau et al , 2020, United States; general care units||Reduce unplanned hospital readmissions using AI-based CDSS||Controlled study|
|Saverino et al , 2021, Italy; rehabilitation||Describe the role of a digital AI platform in facilitating the implementation of changes in rehabilitation service during the COVID-19 pandemic||Retrospective observational study|
|Schlicher et al , 2021, United States; management||Discuss the implementation of data analytics in AI-enabled mission control at one of the largest health care service providers in Washington state||Case study|
|Schuh et al , 2018, Austria; intensive care, oncology, and nephrology||Outline the technical and clinical aspects of 3 CDSSs integrated into practice at Vienna General Hospital||Case study describing 3 projects|
|Semenov et al , 2016, Russia; laboratory||Present research and development of a decision support system for the patients of a laboratory service||Case study|
|Sendak et al , 2020, United States; emergency department||Describe the steps taken to integrate Sepsis Watch, a sepsis detection and management platform, into routine care delivery at Duke University Hospital in Durham, North Carolina||Case study|
|Snowdon et al , 2020, United States; interdisciplinary||Describe the system implemented, workflow changes, and impact on vulnerable citizens||Case study|
|Strohm et al , 2020, The Netherlands; radiology||Identify barriers and facilitators to the implementation of AI applications in clinical radiology||Case study (multiple)|
|Sukums et al , 2015, Ghana and Tanzania; primary care||Describe health workers’ acceptance and use of the CDSS for maternal care at rural facilities in Ghana and Tanzania and identify factors affecting successful adoption||Mixed methods|
|Sun , 2021, China; hospital-wide implementation||Study how social power among various stakeholders affects IT adoption in health care||Mixed methods|
|Tamposis et al , 2022, Greece; urology||Present design and implementation of a software platform for supporting detection as well as using and processing clinical, bio-chemical, imaging, and histopathologic findings from fusion biopsy||Case study|
|Tan et al , 2021, Singapore; radiology||Describe the use of AI for automatic detection and flagging of CT findings not reported by radiologists to improve patient safety||Case study|
|Thurso et al , 2021, Slovakia; dentistry||Evaluate the clinical impact of an AI upgrade of an existing orthodontic mobile coaching app||Pre-post study|
|Wen et al , 2019, United States; hospital-wide implementation||Present recommendations for developing natural language processing tool sets based on the experience of developing clinical natural language processing at the Mayo Clinic in Rochester, Minnesota||Case study|
|Wijnhoven , 2021, The Netherlands; neonatal care||Theory formalization of grounded insights from a CDSS development case, and by doing this create an organizational learning theoretical foundation for AI development in organizations||Case study|
|Wong et al , 2021, Canada; oncology||Characterize the impact of deep learning–based auto-segmented contour models in the clinical workflow at 2 cancer centers||User feedback survey|
aCDSS: clinical decision support system.
bAI: artificial intelligence.
cVTE: venous thromboembolism.
dCT: computed tomography.
Research Question 2: Types and Applications of AI Technology
The most common type of AI application implemented was automation or optimization technology, reported in 71% (32/45) of the implemented systems [, , - , - , , , , , , , , - , - , - ]. Other technologies implemented included human language technologies, computer vision, and robotics technology ( and and ). The most common AI model was a symbolic or knowledge-based model, reported in nearly half (22/45, 49%) of the reviewed studies [ , - , , , - , - , , , ], followed by statistical models (9/45, 20%) [ , - , , , ]. The most commonly performed task was recognition (16/45, 36%) [ , , , - , , , , - , , ], followed by forecasting (9/45, 20%) [ , , - , , , , ]. Other tasks performed were event detection, goal-driven optimization, interaction support, and personalization ( , and ). Although more than half (24/45, 53%) of the AI applications had no action autonomy [ , - , , , , , , , , - , , - , - ], a few reported applications had low (2/21, 10%) [ , ], medium (4/21, 19%) [ , , , ], or high (6/21, 29%) [ , , , , , ] action autonomy ( , and ). Nearly three-quarters of all AI systems were intended for clinical care (33/45, 73%) [ , , , - , , - , - , - ], and the majority (18/33, 55%) of these concerned providing support to inform the patient-provider encounter [ , , , , , , , , , , , , , , , , ], followed by diagnosis and prediction-based diagnosis (13/33, 39%) [ , , , , , , , , , , , , ]. The remaining AI systems (12/45, 27%) were intended for health systems management and planning [ , , , , , - , - ]. Health care providers were the most common target users; most often physicians (19/45, 42%) [ , , , - , - , - , , , , , , , , , , , ], followed by nurses (6/45, 13%) [ , , , , , ]. Other intended users included health workers, technicians, managers, patients or caregivers, and the general public ( and ).
Research Question 3: Implementation Process Characteristics
The research focus in approximately a third of the studies was to present the effectiveness of the implemented intervention (16/45, 36%) [, , - , , , , - , , , , , ]. Other research foci included user experiences [ , , , , , ], AI use metrics [ , , , , ], and identification of barriers or facilitators [ , , , , , ] ( , and ). Most (32/45, 71%) of the studies described the implementation process as successful, and only a few (4/45, 9%) described it as unsuccessful (in the rest of the studies, the success of the implementation was either not mentioned, or the outcome was inconclusive).
In a little more than half (23/45, 51%) of the reviewed studies, the motives behind the implementation were not described. For those studies that did (22/45, 49%), we identified 6 types of motives, with Improve health care quality and Achieve better patient outcomes being the 2 most common. Studies in the former theme described AI systems used to improve quality of services [, , , , ], reduce diagnostic errors [ ], reduce hospital length of stay [ ], or reduce unplanned readmissions [ , ], whereas studies in the latter theme described AI systems used to achieve better patient survival [ , ]. Another theme, Improve efficiency, focused on health care–cost reduction, increased service production, and optimization of public services [ , , , , ]. Respond to the COVID-19 pandemic was stated as a motive necessitated by the need for access to the most up-to-date information [ ], the sudden surge in demand for health care services [ ], prioritization of limited resources [ ], and reorganization of service delivery in response to local guidelines for prevention of infection transmission [ ]. Improve provider satisfaction focused on workload reduction for health care professionals [ , ]. Empower patients by using AI to support interpretations of laboratory investigations, rather than just the test results, was another motive for implementing AI [ ].
Of the 45 included studies, 3 (7%) had an explicit focus on implementation processes [, , ]. In the other studies, characteristics common to implementation processes were identified: cocreation, contextualization, nondisruptive workflow design, communication, learning focus, training, incentives, and organizational strategies. Both barriers and facilitators were described.
Several (8/45, 18%) implementation efforts involved cocreation with multidisciplinary stakeholders, starting from an ideation phase that included problem identification, requirement collection, and design or redesign of clinical workflows to facilitate AI-system integration [, , , , , , , ]. Cocreation also involved end users in the design of user interfaces [ , ]. Contextualization of AI systems relating to the local context and target population was highlighted as important in development and implementation [ , ]. Nondisruptive workflow design was emphasized, where efforts were made to design AI systems around existing roles and functions of the intended user to avoid radical modification of current practice to fit the AI system [ , , ]. Communication efforts were seen as central to building trust and promoting use by sharing evidence of AI effectiveness with clinicians and describing overall benefits of the technology [ , , ], appointing champions to promote AI among peers [ , , ], and encouraging informal communication between clinicians and IT developers to cultivate relationships and build trust in the AI [ ]. However, the study by Sendak et al [ ] encouraged the separation of developers and clinicians and made conscious efforts to shift focus away from the technical aspects of AI. A learning focus could begin in the ideation phase to understand and assess the problem to be addressed by AI before coding, through development and implementation, by iteratively testing and adjusting workflows [ ]. After implementation, learning continued through the continuous capture of user feedback to enable improvement [ ]. Training involved both informal and formal sessions to enable AI use [ , ]. After implementation, training could continue in formal peer-group meetings to share best practices and individual training and support for more reluctant users [ ]. Incentives were used to promote or enforce AI use. More controlling approaches included periodic monitoring and audits [ , ] or removing alternative ways of performing the task altogether to necessitate AI use [ ]. Gamification was used to promote a feeling of reward and competition [ , ]. Organizational efforts involved including the hospital’s top leadership as essential members of the project team and the design and implementation of the AI system to promote uptake [ , ]. One organization formed a special governance committee as a formal mechanism to monitor AI use among health care providers [ ]. Another organization’s innovation strategy included innovation managers as part of the organizational structure to promote AI [ ].
In 7% (3/45) of the studies [, , ], the use of the following implementation frameworks was mentioned: the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework [ ]; the Nonadoption, Abandonment, Scale-up, Spread, and Sustainability framework [ ]; and the Socialization, Externalization, Combination, and Internalization model of knowledge dimensions [ ]. Of the 45 included studies, 4 (9%) proposed new frameworks, principles, or recommendations based on their presented findings and implementation experiences [ , , , ]. Moorman [ ] proposed 6 principles for implementation of AI: elements of trust and transparency, minimal impact on workflows, stakeholder buy-in, relevant education, actionability of AI outputs, and sustainability through follow-up interactions. Reis et al [ ] proposed a framework for overcoming cognitive and affective resistance to AI implementation centered around concerns of users (physicians), such as transparency and understandability of the AI system, involvement of users in the AI training, and trust in the AI system. Sun [ ] proposed a power strategy matrix for AI adoption, suggesting that a “boss strategy” or “expert strategy” can influence adoption. Wen et al [ ] presented 3 desiderata for developing an AI-based platform, where the second one focused on improving adoption.
Our aim with this study was to explore how the implementation of AI in health care practice has been empirically investigated in the research literature. We found that research on implementation of AI systems is mostly published in high-income countries, covers many different clinical settings and disciplines, and predominantly focuses on care providers as users. The AI models are primarily symbolic or knowledge based, use automation or optimization technologies, and are mainly used to perform tasks related to recognition. AI systems are predominantly intended for clinical care, particularly clinical care pertaining to patient-provider encounters. Most possess no action autonomy but rather support human decision-making. The focus of most research is on establishing the effectiveness of interventions or related to technical and computational aspects of AI systems. Focus on the specifics of implementation processes does not yet seem to be a priority in research, and the use of frameworks to guide implementation is rare.
Most of the studies were published very recently (2020-2022), which is unsurprising given the temporal distribution of AI health care studies. Research on AI implementation in health care is predominantly conceptual in nature, dominated by commentaries, perspectives, opinion articles, and conceptual frameworks that raise important questions and issues but without much-needed empirical evidence [- ]. As the empirical evidence base for the implementation of AI solutions in routine health care is still narrow and premature, it limits possibilities for generalization both for practice and for the advancement of methodological approaches. Most of the articles were published in high-income countries, particularly the United States. This finding is consistent with the more developed digital health infrastructure, routine use of electronic health records, and big data initiatives in North American and European countries and aligns with other reviews of AI applications in various fields of health care [ , , ]. The many different clinical settings and disciplines could corroborate the data-driven nature of health care; the fact that AI is highly applicable; or that because of its nascent state, AI is still being tried in many different contexts. Given the focus on clinical care, it is not surprising that the intended users were mostly health care providers, particularly physicians. A recent scoping review on the use of AI in primary care found a similar predominance of physicians as target end users [ ]. This suggests a view of AI systems as tools to support decision-making by physicians rather than other health professionals. It was surprising to find a scarcity of implementations of AI applications to handle infectious diseases (except for the study by McKillop et al [ ]), given the overwhelming attention given to, and funding provided for, the management of the COVID-19 pandemic in 2020-2022. Another underrepresented area where AI holds a strong promise is mental health (except for the studies by Bennett [ ] and Rahman et al [ ]).
Types and Applications of AI Technology
Nearly half of the AI models were symbolic or knowledge based. They used human-generated logical representations, rules, and ontologies to infer conclusions and have greater explainability than models that are based on pure data-driven or statistical approaches. However, they might not live up to the full potential of AI because they are “hard-coded, expert cookbooks” that are limited by the knowledge that is encoded into them . Data-driven, statistical approaches such as machine learning learn predictive functions based on the inputted data. However, these methods are opaque and have implications for health care in relation to patient or provider trust, accountability and quality assurance, and patient safety [ , ]. The World Health Organization’s guidance on ethics and governance of AI for health recognizes the potential trade-off between transparency and accuracy but encourages AI explainability and transparency over black-box approaches [ ]. The predominance of knowledge-based or symbolic models, whose greater transparency and longer existence may ease acceptance among care providers, is in line with previous reviews [ ]. However, the majority of recently published AI models use data-driven or hybrid technologies, and knowledge-based models comprised only a minority of the applications [ ]. Our study found that automation or optimization technologies were by far the most common, followed by human language technologies. More than half of the AI systems implemented had no action autonomy. Instead, they were human decision support systems where the AI system cannot act on its recommendation or output but depends on the human operating the system to use or disregard the recommendation made by it. This finding indicates that decision support systems are the types of AI systems that have achieved adoption the earliest, likely because they enhance human actions and cause minimal disruption to clinical workflows [ ].
This study found that the way the implementation process of AI systems in health care is researched is varied and builds on many types of study designs and methodologies. A little more than half of the included studies did not provide a clear motivation for implementing an AI system, which is a key factor for successful adoption of AI in health care . The lack of a clear motivation indicates poor alignment with well-defined needs from clinical practice and risks reinforcing a technology-focused logic regarding implementation of AI in health care. This observation might reflect the lack of consistent understanding of what is meant by implementation of AI in daily practice and a lack of methodological consistency in how such implementations should be researched and reported. Most of the studies either had a technical or computational understanding of implementation or viewed implementation in terms of the effectiveness of the intervention. There was not much focus on the actual process of implementation studies but more on presenting cases of implementation. This indicates the relatively nascent nature of evidence in this field and is similar to other studies, which highlights that many of the publications on AI in health care focus on the methods and technical aspects of applying the AI model to clinical scenarios but provide very little information on the actual process of its implementation in practice [ , ].
Despite the limited focus in the studies on researching the implementation process, our inductive analysis identified the following implementation elements: cocreation, designing nondisruptive workflows, maintaining a learning focus, communication, contextualization, leadership and conducive organizational structure, trainings, and enforcement or incentivization of AI use. These aspects are not unique to AI but have been highlighted as important interventions for the adoption of all digital technologies, including AI; for example, the involvement of end users in the design and implementation of IT services and applications forms the basis of user-centered design, which is seen as an important driver of uptake of digital technologies . The commitment, involvement, and accountability of leaders is also a well-known factor for successful implementation in practice [ ]. Seamless integration with existing workflows was another factor highlighted as central to adoption of AI systems. This finding is consistent with the fact that most studied cases of AI system implementation were based on decision support systems that have no action autonomy and can be conveniently incorporated into routine workflows. However, it is challenging to draw generalized conclusions on the AI implementation strategies from such systems because they introduce incremental improvements in the workflows and do not represent more disruptive types of AI systems; for example, those with high action autonomy.
The findings of this study corroborate the recent work by Gama et al  regarding the uncertainty of what should be considered AI and the notion that our understanding of implementation is still in the early stages of development. We would add that this understanding is made even more complex by the lack of agreement on what is meant by the term implementation. We rejected numerous studies during the screening because the term implementation was used in a computational sense; for example, the product concept or requirements were implemented as a code, or the coded algorithm was implemented using an existing data set. Even in studies involving real-world settings, the term was used to mean execution of a plan without reflection on the process of execution. The focus of implementation as an intentional effort designed to change routine practice, adapt interventions, or increase the uptake of interventions into routine practice was scarce in the published literature.
Limitations and Methodological Considerations
The strengths of this study include the substantial number of records reviewed and the rigor observed during the screening process. The search strategy was comprehensive and broad, and covered 5 different electronic databases. However, we did not include a broader search of the gray literature that would have undoubtedly captured additional cases and potentially identified more cases representing ongoing or completed implementation projects not yet published in the research literature. As we aimed to investigate the experiences from implementation in clinical practice, during screening we removed clinical trials, case reports, pilots, feasibility studies, and other forms of limited and controlled introduction of AI applications in practice. We expect there to be a lag between the work of technology companies and care providers and subsequent academic publications. However, because of the number of records we identified and the previously found extensive availability of opinion-based articles in the literature in the form of perspectives, insights, and narrative reviews , we made a conscious choice to focus on peer-reviewed articles. Although this procedure might risk excluding relevant knowledge from smaller or unsuccessful implementation attempts or other research adjacent to implementation processes, we delimited the results to the literature based on actual experiences from implementation in everyday clinical practice.
Our initial screening of title and abstracts did not require decisions by 2 reviewers, but all decisions in the full-text screening were confirmed in pairs. We deliberately worked to maintain consistency and mitigate individual variation through biweekly meetings where we worked to establish a psychologically safe environment that encouraged all authors to raise or flag doubts, discuss the application of exclusion criteria, or consider differing interpretations. When in doubt, we would backtrack or repeat without blame, and all conflicts and uncertainties were resolved through discussion until consensus was reached. Additional meetings were held with other experts in the domain to ensure methodological rigor. Although the Arksey and O’Malley framework for scoping reviews  does not include a quality appraisal, we would recommend that future authors consider doing so as the number of articles that carefully consider implementation increases.
The current body of empirical evidence demonstrates a dissonance between research and practice needs. On the one hand, conceptual and methodological AI research builds on large promises of AI to revolutionize health care and problematizes its slow uptake into practice. On the other hand, the current empirically supported knowledge derives mostly from implementations of AI systems with low action autonomy and highlights lessons on the implementation process that are typical of implementations of other types of information systems. Further research is needed on the more disruptive types of AI systems being implemented in routine care to identify those aspects of implementation unique to AI. This highlights the need for future research to advance in two main streams: (1) to empirically study the implementation processes of various types of AI systems in health care practice and (2) to support empirical research and practical implementations by developing and disseminating an AI-specific implementation framework that would take into account some of the unique aspects related to uptake of AI in health care, such as building trust, addressing transparency issues, developing explainable and interpretable solutions, and addressing ethical concerns around privacy and data protection.
The authors would like to thank Per Nilsen and Julie Reed for contributing critical input to the final draft of the manuscript. This work was financially supported by the Knowledge Foundation. The funders had no involvement in the study design; in the collection, analysis, and interpretation of the data; in the writing of the report; or in the decision to submit the paper for publication.
All authors participated in the design of the study. IL, CS, and JMN retrieved the records from databases. IL, CS, JMN, MS, and MN participated in the screening process. MS, MN, and IL extracted the data. All authors were involved in the analysis and interpretation of data. MS originally drafted the manuscript as a master’s thesis, with CS and IL as supervisors and input from all authors. All authors thereafter drafted and revised the manuscript and approved the final version.
Conflicts of Interest
Detailed search strategy for the study.DOCX File , 29 KB
Overview of articles included in the scoping review (N=45).DOCX File , 21 KB
Types of artificial intelligence (AI) systems implemented and main focus of research. (A) Types of AI technologies implemented, classified according to the Organisation for Economic Co-operation and Development Framework. (B) Types of tasks performed by AI in health care across the included studies. (C) Level of action autonomy in the AI implemented. (D) Overall focus of the paper and results.PNG File , 128 KB
- No longer science fiction, AI and robotics are transforming healthcare. PwC. URL: https://www.pwc.com/gx/en/industries/healthcare/publications/ai-robotics-new-health/transforming-healthcare.html [accessed 2022-05-30]
- Das R. Five Technologies That Will Disrupt Healthcare By 2020. Forbes. 2016 Mar 30. URL: https://www.forbes.com/sites/reenitadas/2016/03/30/top-5-technologies-disrupting-healthcare-by-2020/ [accessed 2022-05-30]
- Bitterman DS, Aerts HJ, Mak RH. Approaching autonomy in medical artificial intelligence. Lancet Digit Health 2020 Sep;2(9):e447-e449 [FREE Full text] [CrossRef] [Medline]
- Langlotz CP. Will artificial intelligence replace radiologists? Radiol Artif Intell 2019 May;1(3):e190058 [FREE Full text] [CrossRef] [Medline]
- Shuaib A, Arian H, Shuaib A. The increasing role of artificial intelligence in health care: will robots replace doctors in the future? Int J Gen Med 2020 Oct 19;13:891-896 [FREE Full text] [CrossRef] [Medline]
- Chen M, Decary M. Artificial intelligence in healthcare: an essential guide for health leaders. Healthc Manage Forum 2020 Jan;33(1):10-18. [CrossRef] [Medline]
- Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol 2017 Dec;2(4):230-243 [FREE Full text] [CrossRef] [Medline]
- Rajpurkar P, Chen E, Banerjee O, Topol EJ. AI in health and medicine. Nat Med 2022 Jan;28(1):31-38. [CrossRef] [Medline]
- Secinaro S, Calandra D, Secinaro A, Muthurangu V, Biancone P. The role of artificial intelligence in healthcare: a structured literature review. BMC Med Inform Decis Mak 2021 Apr 10;21(1):125 [FREE Full text] [CrossRef] [Medline]
- Loftus TJ, Tighe PJ, Filiberto AC, Efron PA, Brakenridge SC, Mohr AM, et al. Artificial intelligence and surgical decision-making. JAMA Surg 2020 Feb 01;155(2):148-158 [FREE Full text] [CrossRef] [Medline]
- Lopez-Jimenez F, Attia Z, Arruda-Olson AM, Carter R, Chareonthaitawee P, Jouni H, et al. Artificial intelligence in cardiology: present and future. Mayo Clin Proc 2020 May;95(5):1015-1039. [CrossRef] [Medline]
- Abbasgholizadeh Rahimi S, Légaré F, Sharma G, Archambault P, Zomahoun HT, Chandavong S, et al. Application of artificial intelligence in community-based primary health care: systematic scoping review and critical appraisal. J Med Internet Res 2021 Sep 03;23(9):e29839 [FREE Full text] [CrossRef] [Medline]
- Checcucci E, Autorino R, Cacciamani GE, Amparore D, De Cillis S, Piana A, Uro-technology and SoMe Working Group of the Young Academic Urologists Working Party of the European Association of Urology. Artificial intelligence and neural networks in urology: current clinical applications. Minerva Urol Nefrol 2020 Feb;72(1):49-57 [FREE Full text] [CrossRef] [Medline]
- Dutt S, Sivaraman A, Savoy F, Rajalakshmi R. Insights into the growing popularity of artificial intelligence in ophthalmology. Indian J Ophthalmol 2020 Jul;68(7):1339-1346 [FREE Full text] [CrossRef] [Medline]
- He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nat Med 2019 Jan;25(1):30-36 [FREE Full text] [CrossRef] [Medline]
- Chen ZH, Lin L, Wu CF, Li CF, Xu RH, Sun Y. Artificial intelligence for assisting cancer diagnosis and treatment in the era of precision medicine. Cancer Commun (Lond) 2021 Nov;41(11):1100-1115 [FREE Full text] [CrossRef] [Medline]
- Grzybowski A, Brona P, Lim G, Ruamviboonsuk P, Tan G, Abramoff M, et al. Artificial intelligence for diabetic retinopathy screening: a review. Eye (Lond) 2020 Mar;34(3):451-460 [FREE Full text] [CrossRef] [Medline]
- Ozsahin I, Sekeroglu B, Musa MS, Mustapha MT, Uzun Ozsahin D. Review on diagnosis of COVID-19 from chest CT images using artificial intelligence. Comput Math Methods Med 2020 Sep 26;2020:9756518 [FREE Full text] [CrossRef] [Medline]
- Romm EL, Tsigelny IF. Artificial intelligence in drug treatment. Annu Rev Pharmacol Toxicol 2020 Jan 06;60:353-369. [CrossRef] [Medline]
- Ho D. Artificial intelligence in cancer therapy. Science 2020 Feb 28;367(6481):982-983. [CrossRef] [Medline]
- Schwalbe N, Wahl B. Artificial intelligence and the future of global health. Lancet 2020 May 16;395(10236):1579-1586 [FREE Full text] [CrossRef] [Medline]
- Lorenzi NM, Riley RT, Blyth AJ, Southon G, Dixon BJ. Antecedents of the people and organizational aspects of medical informatics: review of the literature. J Am Med Inform Assoc 1997;4(2):79-93 [FREE Full text] [CrossRef] [Medline]
- Kukafka R, Johnson SB, Linfante A, Allegrante JP. Grounding a new information technology implementation framework in behavioral science: a systematic analysis of the literature on IT use. J Biomed Inform 2003 Jun;36(3):218-227 [FREE Full text] [CrossRef] [Medline]
- Gooding P, Kariotis T. Ethics and law in research on algorithmic and data-driven technology in mental health care: scoping review. JMIR Ment Health 2021 Jun 10;8(6):e24668 [FREE Full text] [CrossRef] [Medline]
- Čartolovni A, Tomičić A, Lazić Mosler E. Ethical, legal, and social considerations of AI-based medical decision-support tools: a scoping review. Int J Med Inform 2022 May;161:104738. [CrossRef] [Medline]
- Beil M, Proft I, van Heerden D, Sviri S, van Heerden PV. Ethical considerations about artificial intelligence for prognostication in intensive care. Intensive Care Med Exp 2019 Dec 10;7(1):70 [FREE Full text] [CrossRef] [Medline]
- Murphy K, Di Ruggiero E, Upshur R, Willison DJ, Malhotra N, Cai JC, et al. Artificial intelligence for good health: a scoping review of the ethics literature. BMC Med Ethics 2021 Feb 15;22(1):14 [FREE Full text] [CrossRef] [Medline]
- Siala H, Wang Y. SHIFTing artificial intelligence to be responsible in healthcare: a systematic review. Soc Sci Med 2022 Mar;296:114782 [FREE Full text] [CrossRef] [Medline]
- Trocin C, Mikalef P, Papamitsiou Z, Conboy K. Responsible AI for digital health: a synthesis and a research agenda. Inf Syst Front (forthcoming) 2021 Jun 26 [FREE Full text] [CrossRef]
- Fernandes M, Vieira SM, Leite F, Palos C, Finkelstein S, Sousa JM. Clinical decision support systems for triage in the emergency department using intelligent systems: a review. Artif Intell Med 2020 Jan;102:101762. [CrossRef] [Medline]
- Choudhury A, Asan O. Role of artificial intelligence in patient safety outcomes: systematic literature review. JMIR Med Inform 2020 Jul 24;8(7):e18599 [FREE Full text] [CrossRef] [Medline]
- Yin J, Ngiam KY, Teo HH. Role of artificial intelligence applications in real-life clinical practice: systematic review. J Med Internet Res 2021 Apr 22;23(4):e25759 [FREE Full text] [CrossRef] [Medline]
- Wolff J, Pauling J, Keck A, Baumbach J. The economic impact of artificial intelligence in health care: systematic review. J Med Internet Res 2020 Feb 20;22(2):e16866 [FREE Full text] [CrossRef] [Medline]
- de Hond AA, Leeuwenberg AM, Hooft L, Kant IM, Nijman SW, van Os HJ, et al. Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review. NPJ Digit Med 2022 Jan 10;5(1):2 [FREE Full text] [CrossRef] [Medline]
- Mirbabaie M, Stieglitz S, Frick NR. Artificial intelligence in disease diagnostics: a critical review and classification on the current state of research guiding future direction. Health Technol 2021 May 10;11(4):693-731 [FREE Full text] [CrossRef]
- Zidaru T, Morrow EM, Stockley R. Ensuring patient and public involvement in the transition to AI-assisted mental health care: a systematic scoping review and agenda for design justice. Health Expect 2021 Aug;24(4):1072-1124 [FREE Full text] [CrossRef] [Medline]
- Gama F, Tyskbo D, Nygren J, Barlow J, Reed J, Svedberg P. Implementation frameworks for artificial intelligence translation into health care practice: scoping review. J Med Internet Res 2022 Jan 27;24(1):e32215 [FREE Full text] [CrossRef] [Medline]
- Alhashmi SF, Alshurideh M, Al Kurdi B, Salloum SA. A systematic review of the factors affecting the artificial intelligence implementation in the health care sector. In: Proceedings of the 2020 International Conference on Artificial Intelligence and Computer Vision. 2020 Presented at: AICV '20; April 8–10, 2020; Cairo, Egypt p. 37-49 URL: https://link.springer.com/chapter/10.1007/978-3-030-44289-7_4 [CrossRef]
- Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005 Feb;8(1):19-32 [FREE Full text] [CrossRef]
- Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018 Oct 02;169(7):467-473 [FREE Full text] [CrossRef] [Medline]
- Svedberg P, Reed J, Nilsen P, Barlow J, Macrae C, Nygren J. Toward successful implementation of artificial intelligence in health care practice: protocol for a research program. JMIR Res Protoc 2022 Mar 09;11(3):e34920 [FREE Full text] [CrossRef] [Medline]
- Recommendation of the Council on Artificial Intelligence (OECD Legal Instruments. OECD/ LEGAL/O449). Organisation for Economic Co-operation and Development. 2019 May 22. URL: https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449#mainText [accessed 2022-02-26]
- Ethics and Governance of Artificial Intelligence for Health: WHO guidance. World Health Organization. 2021 Jun 28. URL: http://apps.who.int/bookorders [accessed 2022-02-16]
- Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
- Ng R, Tan KB. Implementing an individual-centric discharge process across Singapore public hospitals. Int J Environ Res Public Health 2021 Aug 17;18(16):8700 [FREE Full text] [CrossRef] [Medline]
- Sendak MP, Ratliff W, Sarro D, Alderton E, Futoma J, Gao M, et al. Real-world integration of a sepsis deep learning technology into routine clinical care: implementation study. JMIR Med Inform 2020 Jul 15;8(7):e15182 [FREE Full text] [CrossRef] [Medline]
- Kashyap S, Morse KE, Patel B, Shah NH. A survey of extant organizational and computational setups for deploying predictive models in health systems. J Am Med Inform Assoc 2021 Oct 12;28(11):2445-2450 [FREE Full text] [CrossRef] [Medline]
- McKillop M, South B, Preininger A, Mason M, Jackson G. Leveraging conversational technology to answer common COVID-19 questions. J Am Med Inform Assoc 2021 Mar 18;28(4):850-855 [FREE Full text] [CrossRef] [Medline]
- Moorman LP. Principles for real-world implementation of bedside predictive analytics monitoring. Appl Clin Inform 2021 Aug;12(4):888-896 [FREE Full text] [CrossRef] [Medline]
- Baxter SL, Bass JS, Sitapati AM. Barriers to implementing an artificial intelligence model for unplanned readmissions. ACI open 2020 Jul;4(2):e108-e113 [FREE Full text] [CrossRef] [Medline]
- Romero-Brufau S, Wyatt KD, Boyum P, Mickelson M, Moore M, Cognetta-Rieke C. A lesson in implementation: a pre-post study of providers' experience with artificial intelligence-based clinical decision support. Int J Med Inform 2020 May;137:104072. [CrossRef] [Medline]
- Lai L, Wittbold K, Dadabhoy FZ, Sato R, Landman AB, Schwamm LH, et al. Digital triage: novel strategies for population health management in response to the COVID-19 pandemic. Healthc (Amst) 2020 Dec;8(4):100493 [FREE Full text] [CrossRef] [Medline]
- Strohm L, Hehakaya C, Ranschaert ER, Boon WP, Moors EH. Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors. Eur Radiol 2020 Oct;30(10):5525-5532 [FREE Full text] [CrossRef] [Medline]
- Romero-Brufau S, Wyatt KD, Boyum P, Mickelson M, Moore M, Cognetta-Rieke C. Implementation of artificial intelligence-based clinical decision support to reduce hospital readmissions at a regional hospital. Appl Clin Inform 2020 Aug;11(4):570-577 [FREE Full text] [CrossRef] [Medline]
- Reis L, Maier C, Mattke J, Creutzenberg M, Weitzel T. Addressing user resistance would have prevented a healthcare AI project failure. MIS Q Exec 2020 Dec;19(4):279-296 [FREE Full text] [CrossRef]
- Sun TQ. Adopting artificial intelligence in public healthcare: the effect of social power and learning algorithms. Int J Environ Res Public Health 2021 Dec 01;18(23):12682 [FREE Full text] [CrossRef] [Medline]
- Wijnhoven F. Organizational learning for intelligence amplification adoption: lessons from a clinical decision support system adoption project. Inf Syst Front 2021 Oct 09;24(3):731-744 [FREE Full text] [CrossRef]
- Thurzo A, Kurilová V, Varga I. Artificial intelligence in orthodontic smart application for treatment coaching and its impact on clinical performance of patients monitored with AI-TeleHealth system. Healthcare (Basel) 2021 Dec 07;9(12):1695 [FREE Full text] [CrossRef] [Medline]
- Gonçalves LS, Amaro ML, Romero AD, Schamne FK, Fressatto JL, Bezerra CW. Implementation of an artificial intelligence algorithm for sepsis detection. Rev Bras Enferm 2020 Apr 9;73(3):e20180421 [FREE Full text] [CrossRef] [Medline]
- Damoah IS, Ayakwah A, Tingbani I. Artificial intelligence (AI)-enhanced medical drones in the healthcare supply chain (HSC) for sustainability development: a case study. J Clean Prod 2021 Dec;328:129598. [CrossRef]
- Lacey G, Zhou J, Li X, Craven C, Gush C. The impact of automatic video auditing with real-time feedback on the quality and quantity of handwash events in a hospital setting. Am J Infect Control 2020 Feb;48(2):162-166 [FREE Full text] [CrossRef] [Medline]
- Saverino A, Baiardi P, Galata G, Pedemonte G, Vassallo C, Pistarini C. The challenge of reorganizing rehabilitation services at the time of COVID-19 pandemic: a new digital and artificial intelligence platform to support team work in planning and delivering safe and high quality care. Front Neurol 2021 Apr 29;12:643251 [FREE Full text] [CrossRef] [Medline]
- Wong J, Huang V, Wells D, Giambattista J, Giambattista J, Kolbeck C, et al. Implementation of deep learning-based auto-segmentation for radiotherapy planning structures: a workflow study at two cancer centers. Radiat Oncol 2021 Jun 08;16(1):101 [FREE Full text] [CrossRef] [Medline]
- Herman B, Sirichokchatchawan W, Nantasenamat C, Pongpanich S. Artificial intelligence in overcoming rifampicin resistant-screening challenges in Indonesia: a qualitative study on the user experience of CUHAS-ROBUST. J Health Res (forthcoming) 2021 Jun 15 [FREE Full text] [CrossRef]
- Chong J, Curtain C, Gad F, Passam F, Soo G, Levy R, et al. Development and implementation of venous thromboembolism stewardship across a hospital network. Int J Med Inform 2021 Nov;155:104575. [CrossRef] [Medline]
- Tan JR, Cheong EH, Chan LP, Tham WP. Implementation of an artificial intelligence-based double read system in capturing pulmonary nodule discrepancy in CT studies. Curr Probl Diagn Radiol 2021;50(2):119-122. [CrossRef] [Medline]
- Petitgand C, Motulsky A, Denis JL, Régis C. Investigating the barriers to physician adoption of an artificial intelligence- based decision support system in emergency care: an interpretative qualitative study. Stud Health Technol Inform 2020 Jun 16;270:1001-1005. [CrossRef] [Medline]
- Chonde DB, Pourvaziri A, Williams J, McGowan J, Moskos M, Alvarez C, et al. RadTranslate: an artificial intelligence-powered intervention for urgent imaging to enhance care equity for patients with limited English proficiency during the COVID-19 pandemic. J Am Coll Radiol 2021 Jul;18(7):1000-1008 [FREE Full text] [CrossRef] [Medline]
- García Bermúdez I, González Manso M, Sánchez Sánchez E, Rodríguez Hita A, Rubio Rubio M, Suárez Fernández C. Usefulness and acceptance of telephone monitoring by a virtual assistant for patients with COVID-19 following discharge. Rev Clin Esp (Barc) 2021 Oct;221(8):464-467 [FREE Full text] [CrossRef] [Medline]
- Tamposis I, Tsougos I, Karatzas A, Vassiou K, Vlychou M, Tzortzis V. PCaGuard: a software platform to support optimal management of prostate cancer. Appl Clin Inform 2022 Jan;13(1):91-99 [FREE Full text] [CrossRef] [Medline]
- Mohamed M, Ferguson DJ, Venugopal A, Alam MK, Makki L, Vaid NR. An artificial intelligence based referral application to optimize orthodontic referrals in a public oral healthcare system. Semin Orthod 2021 Jun 1;27(2):157-163. [CrossRef]
- Morales HM, Guedes M, Silva JS, Massuda A. COVID-19 in Brazil-preliminary analysis of response supported by artificial intelligence in municipalities. Front Digit Health 2021 Jun 17;3:648585 [FREE Full text] [CrossRef] [Medline]
- Davis MA, Rao B, Cedeno PA, Saha A, Zohrabian VM. Machine learning and improved quality metrics in acute intracranial hemorrhage by noncontrast computed tomography. Curr Probl Diagn Radiol 2022;51(4):556-561. [CrossRef] [Medline]
- Snowdon JL, Robinson B, Staats C, Wolsey K, Sands-Lincoln M, Strasheim T, et al. Empowering caseworkers to better serve the most vulnerable with a cloud-based care management solution. Appl Clin Inform 2020 Aug;11(4):617-621 [FREE Full text] [CrossRef] [Medline]
- Schlicher J, Metsker M, Shah H, Demirkan H. From NASA to healthcare: real-time data analytics (mission control) is reshaping healthcare services. Perspect Health Inf Manag 2021 Oct 1;18(4):1g [FREE Full text] [Medline]
- O'Neill TJ, Xi Y, Stehel E, Browning T, Ng YS, Baker C, et al. Active reprioritization of the reading worklist using artificial intelligence has a beneficial effect on the turnaround time for interpretation of head CT with intracranial hemorrhage. Radiol Artif Intell 2020 Nov 18;3(2):e200024 [FREE Full text] [CrossRef] [Medline]
- Bennett CC. Clinical productivity system - a decision support model. Int J Prod Performance Manag 2011;60(3):311-319 [FREE Full text] [CrossRef]
- Schuh C, de Bruin JS, Seeling W. Clinical decision support systems at the Vienna general hospital using Arden syntax: design, implementation, and integration. Artif Intell Med 2018 Nov;92:24-33. [CrossRef] [Medline]
- Anand V, Carroll AE, Biondich PG, Dugan TM, Downs SM. Pediatric decision support using adapted Arden syntax. Artif Intell Med 2018 Nov;92:15-23 [FREE Full text] [CrossRef] [Medline]
- Wen A, Fu S, Moon S, El Wazir M, Rosenbaum A, Kaggal VC, et al. Desiderata for delivering NLP to accelerate healthcare AI advancement and a Mayo Clinic NLP-as-a-service implementation. NPJ Digit Med 2019 Dec 17;2:130 [FREE Full text] [CrossRef] [Medline]
- Rath S, Rajaram K, Mahajan A. Integrated anesthesiologist and room scheduling for surgeries: methodology and application. Operation Res 2017 Jul 20;65(6):1460-1478 [FREE Full text] [CrossRef]
- Rais A, Alvelos F, Figueiredo J, Nobre A. Optimization of logistics services in hospitals. Intl Trans Op Res 2018 Jan;25(1):111-132 [FREE Full text] [CrossRef]
- Dios M, Molina-Pariente JM, Fernandez-Viagas V, Andrade-Pineda JL, Framinan JM. A decision support system for operating room scheduling. Comput Ind Eng 2015 Oct;88(C):430-443 [FREE Full text] [CrossRef]
- Litvin CB, Ornstein SM, Wessell AM, Nemeth LS, Nietert PJ. Adoption of a clinical decision support system to promote judicious use of antibiotics for acute respiratory infections in primary care. Int J Med Inform 2012 Aug;81(8):521-526. [CrossRef] [Medline]
- Cruz NP, Canales L, Muñoz JG, Pérez B, Arnott I. Improving adherence to clinical pathways through natural language processing on electronic medical records. Stud Health Technol Inform 2019 Aug 21;264:561-565. [CrossRef] [Medline]
- Semenov I, Kopanitsa G. Development of a clinical decision support system for the patients of a laboratory service. Stud Health Technol Inform 2016;228:90-94. [Medline]
- Campion Jr TR, Waitman LR, Lorenzi NM, May AK, Gadd CS. Barriers and facilitators to the use of computer-based intensive insulin therapy. Int J Med Inform 2011 Dec;80(12):863-871 [FREE Full text] [CrossRef] [Medline]
- Kalil AJ, Dias VM, Rocha CD, Morales HM, Fressatto JL, Faria RA. Sepsis risk assessment: a retrospective analysis after a cognitive risk management robot (Robot Laura®) implementation in a clinical-surgical unit. Res Biomed Eng 2018 Nov 22;34(4):310-316 [FREE Full text] [CrossRef]
- Sukums F, Mensah N, Mpembeni R, Massawe S, Duysburgh E, Williams A, et al. Promising adoption of an electronic clinical decision support system for antenatal and intrapartum care in rural primary healthcare facilities in sub-Saharan Africa: the QUALMAT experience. Int J Med Inform 2015 Sep;84(9):647-657. [CrossRef] [Medline]
- Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999 Sep;89(9):1322-1327. [CrossRef] [Medline]
- Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A'Court C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res 2017 Nov 01;19(11):e367 [FREE Full text] [CrossRef] [Medline]
- Nonaka I. A dynamic theory of organizational knowledge creation. Organ Sci 1994 Feb;5(1):14-37 [FREE Full text] [CrossRef]
- Hadley TD, Pettit RW, Malik T, Khoei AA, Salihu HM. Artificial intelligence in global health -a framework and strategy for adoption and sustainability. Int J MCH AIDS 2020;9(1):121-127 [FREE Full text] [CrossRef] [Medline]
- Arnold MH. Teasing out artificial intelligence in medicine: an ethical critique of artificial intelligence and machine learning in medicine. J Bioeth Inq 2021 Mar;18(1):121-139 [FREE Full text] [CrossRef] [Medline]
- Reddy S, Allan S, Coghlan S, Cooper P. A governance model for the application of AI in health care. J Am Med Inform Assoc 2020 Mar 01;27(3):491-497 [FREE Full text] [CrossRef] [Medline]
- Higgins D, Madai VI. From bit to bedside: a practical framework for artificial intelligence product development in healthcare. Adv Intell Syst 2020 Oct;2(10):2000052 [FREE Full text] [CrossRef]
- Wahl B, Cossy-Gantner A, Germann S, Schwalbe NR. Artificial intelligence (AI) and global health: how can AI contribute to health in resource-poor settings? BMJ Glob Health 2018 Aug 29;3(4):e000798 [FREE Full text] [CrossRef] [Medline]
- Peiffer-Smadja N, Rawson TM, Ahmad R, Buchard A, Georgiou P, Lescure FX, et al. Machine learning for clinical decision support in infectious diseases: a narrative review of current applications. Clin Microbiol Infect 2020 May;26(5):584-595 [FREE Full text] [CrossRef] [Medline]
- Kueper JK, Terry AL, Zwarenstein M, Lizotte DJ. Artificial intelligence and primary care research: a scoping review. Ann Fam Med 2020 May;18(3):250-258 [FREE Full text] [CrossRef] [Medline]
- Rahman MM, Khatun F, Uzzaman A, Sami SI, Bhuiyan MA, Kiong TS. A comprehensive study of artificial intelligence and machine learning approaches in confronting the coronavirus (COVID-19) pandemic. Int J Health Serv 2021 Oct;51(4):446-461 [FREE Full text] [CrossRef] [Medline]
- Quinn TP, Jacobs S, Senadeera M, Le V, Coghlan S. The three ghosts of medical AI: can the black-box present deliver? Artif Intell Med 2022 Feb;124:102158. [CrossRef] [Medline]
- Kort om Artificiell intelligens i hälso- och sjukvården. Statens medicinsk-etiska råd. 2020. URL: https://smer.se/2020/03/31/kort-om-artificiell-intelligens-i-halso-och-sjukvarden/ [accessed 2022-05-30]
- Wang Y, Wang L, Rastegar-Mojarad M, Moon S, Shen F, Afzal N, et al. Clinical information extraction applications: a literature review. J Biomed Inform 2018 Jan;77:34-49 [FREE Full text] [CrossRef] [Medline]
- Montani S, Striani M. Artificial intelligence in clinical decision support: a focused literature survey. Yearb Med Inform 2019 Aug;28(1):120-127 [FREE Full text] [CrossRef] [Medline]
- Saria S. Five keys for using artificial intelligence in care delivery, and five common mistakes made in evaluating AI tools. Not All AI Is Created Equal: Strategies for Safe and Effective Adoption. NEJM Catalyst. 2022 Mar 23. URL: https://catalyst.nejm.org/doi/full/10.1056/CAT.22.0075 [accessed 2022-05-30]
- Bernstein ML, McCreless T, Côté MJ. Five constants of information technology adoption in healthcare. Hosp Top 2007;85(1):17-25. [CrossRef] [Medline]
- Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009 Aug 07;4:50 [FREE Full text] [CrossRef] [Medline]
|AI: artificial intelligence|
|PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews|
Edited by G Eysenbach; submitted 12.06.22; peer-reviewed by M Gordon, م قادرزاده, R Damaševičius; comments to author 01.07.22; revised version received 19.08.22; accepted 30.08.22; published 05.10.22Copyright
©Malvika Sharma, Carl Savage, Monika Nair, Ingrid Larsson, Petra Svedberg, Jens M Nygren. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 05.10.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.