The use of AI in health care can be characterized as a complex intervention comprising a number of different elements that act both independently and interdependently. These include technological (eg, functionality and user interface), organizational (eg, implementation process, including training and support), and social components (eg, staff attitudes). Studying complex interventions requires a strong theoretical foundation [15]. Realist evaluation is a theory-driven approach to understanding for whom and in what circumstances complex interventions work [16] and has been used for studying a number of complex interventions, including health information technology [17]. Realist approaches can be used both to evaluate complex interventions and inform intervention design [18]. For these reasons, a realist approach is highly appropriate for the conduct of this review.

Technology depends on human agency to work; technology in and of itself does not cause change, it is how people choose to make use of (or not) the resources that a technology offers to them that lead to what we typically consider to be the impacts of technology. Such choices are highly dependent on context. So, while a technology may provide the desired impact in one context, it is unlikely to produce the same impact across all settings. Realist evaluation is a methodology that explicitly recognizes this. It involves constructing, testing, and refining stakeholders’ ideas or theories about how and in what contexts a technology is supposed to work. These theories detail how particular contexts shape users’ responses to components of the technology (intervention mechanisms) to generate outcomes. They are presented as context-mechanism-outcome (CMO) configurations, where context (C) + mechanism (M) = outcome (O). In this way, and in comparison to more general qualitative approaches, realist evaluation moves beyond listing barriers and facilitators, offering specificity in understanding the relationship between contexts, mechanisms, and outcomes.

The elicitation of stakeholders’ theories can be done in a number of ways, such as interviewing stakeholders, reviewing the existing literature on the topic, identifying relevant theories from the sociological or other works of literature, or some combination of these approaches. To ensure we are building on existing work, we have chosen to elicit stakeholders’ theories through a review of the literature related to the use of AI in pathology. In contrast to a full realist review, where published evidence is used to test and refine stakeholders’ theories [19], theories elicited in this review will be refined through interviews with pathologists in later stages of the research.

While this is not a full realist review, where relevant, we have reported the items included in the Realist and Meta-narrative Evidence Synthesis: Evolving Standards (RAMESES) reporting guidelines for the reporting of realist reviews [20] (see Multimedia Appendix 1 for completed RAMESES checklist).

The overriding question for the review was what works, for whom, in what circumstances, and how to encourage uptake and impact of AI in pathology. To address this question, several searches were undertaken. Search 1 sought studies, reports, and policy documents from the following databases: arXiv.org (Cornell University) repository, Ovid MEDLINE(R), and HMIC Health Management Information Consortium (Ovid). Searches were developed for the concepts of artificial intelligence and histopathology. Subject headings and free text words were identified for use in the search concepts by an information specialist (JW) and project team members. Further terms were identified and tested from known relevant papers. Search results were limited to English-language publications. We also limited search results to publications published since 2000, given the recent development of AI in pathology. The searches were peer-reviewed by a second information specialist using the Peer Review of Electronic Search Strategies (PRESS) checklist (see Multimedia Appendix 2 for full search strategies) [21].

Search 2 sought reports discussing AI, authored by Eric Topol in Ovid MEDLINE(R), Sciences Citation Index (Clarivate Analytics Web of Science), and Emerging Sources Citation Index (Clarivate Analytics Web of Science; see Multimedia Appendix 3). While searching for a particular author is not typically used in traditional systematic reviews, named author searches are recognized as a method for increasing the number of outputs included in a review [22]. Such an approach is recommended for realist reviews [23] and more generally for understanding the theories that underpin complex interventions and the contexts in which they are implemented [22]. Named author searches are used in the theory elicitation stage of realist reviews as a means of gathering opinion pieces, remembering that we are looking for theories rather than empirical evidence [24]. Eric Topol was chosen as an “opinion leader,” given his popular science book on the topic of AI in health care [25] and his input into England’s strategy on digital health training and education, including around AI [26].

Searches were also undertaken of the following websites: the Food and Drugs Administration, the College of American Pathologists, the Royal College of Pathologists, and the Digital Pathology Association along with a number of Google searches (see Multimedia Appendix 4). Additional papers were identified through personal recommendation and “snowballing” (pursuing references of references) [27]. Search results were collated and deduplicated in EndNote (Clarivate).

As the aim of this review is to identify and characterize stakeholders’ perspectives regarding contextual factors that will enhance or restrict the uptake of AI in pathology rather than to assess the validity of these perspectives, the identified papers were screened not based on rigor but on relevance to the review question. Titles and abstracts returned from the search results were screened in EndNote, asking the following questions: is the paper about AI in pathology or health care? And does the paper contain ideas about how, for whom, and in what circumstance AI can work (in the clinical setting)? Full-text copies of potentially relevant documents were then obtained and read by the reviewers to identify if they contained ideas about the introduction of AI into pathology or other relevant health care settings.

While these searches were primarily concerned with pathology, papers from outside this field (particularly radiology) were also returned. This is due to the medical use of “pathology” to describe the features typical of the way a disease presents, in addition to it being the branch of medicine that deals with the analysis of body tissue for diagnostic or forensic purposes. These tangential papers were retained as they potentially contained concepts that were transferable or relevant to the field of pathology. Similarly, a number of papers concerned with image analysis without the use of AI were retained on the basis that some of the potential supports and constraints to implementation they described would have relevance to the use of AI-based image analysis.

Documents were entered into NVivo 12 (QSR International Pty Ltd) software for qualitative analysis. Sections of text were indexed in an iterative process using a series of codes that evolved to represent topics relevant to the review question. Following the realist strategy, these codes sought to capture different contexts, mechanisms, and outcomes that could affect the introduction of AI in pathology. The coded data were used to produce narrative summaries of each of the identified contexts, mechanisms, and outcomes. The reviewers and other authors then discussed these narratives and translated them into CMO configurations.

While the documents provided data about outcomes associated with the use of AI and some data could be drawn out about contexts, there was little about the mechanisms through which these were achieved. Therefore, to guide our thinking, we also drew on substantive theories concerning the implementation of technology and complex interventions more generally, identifying potentially relevant theories through team discussions of the emerging themes. Use of substantive theory is in line with the realist approach, which argues that the design of interventions tends to be based on a limited number of theories regarding human behavior and therefore, rather than starting from scratch when evaluating a new intervention, researchers should also make use of existing theory [16].

The search identified 1420 papers, 4 web pages, and 9 government or institution or foundation documents, providing a total of 1433 unique records (see Multimedia Appendix 5 for PRISMA [Preferred Reporting Items for Systematic Reviews and Meta-Analyses] diagram). After title and abstract screening, 1294 documents were determined to not be relevant, leaving 139 potentially relevant documents. After screening of full texts of these, 101 documents were identified as relevant. All 101 documents were coded, although there was much repetition of the themes contained within them. Below we summarize these themes, organizing them according to the realist concepts of context, mechanism, and outcome. As realist evaluation typically starts with looking at outcome patterns before identifying the contexts and mechanisms that lead to the outcome pattern, the anticipated impacts (outcomes), both positive and negative, of AI in pathology were first considered. The contexts that may impact uptake and impacts of AI were then considered. The mechanisms that may be triggered in particular contexts for the outcomes to be achieved were finally considered. The analysis also suggested practical challenges of introducing AI, such as infrastructure and the need for adequate training data, but we do not discuss these in what follows as our focus is on contextual factors that are likely to shape pathologists’ responses to AI.

In realist evaluation, mechanisms are understood as a combination of the resources that a technology provides and the users’ responses to those resources. The literature described a broad range of actual and potential technologies that could be considered as resources. These different technologies can be conceptualized at a more abstract level, using a typology that we identified in the literature. This typology describes 3 different models of the relationship between the pathologist and AI [28].

In the first model, the pathologist is “in the loop,” or what some refer to as the “augmented pathologist” [56], where AI is a tool that pathologists use to aid their diagnosis [3,31,33,35]. It has been proposed that this is the model that we are most likely to see in the short term. Some have suggested that AI should be seen as a colleague who can provide second opinions on difficult cases, but without individual challenges, such as tiredness, and collaborative challenges, such as those that result from hierarchies [36]. The AI-human combination is believed to be more accurate because the errors made by AI are not strongly correlated with the errors made by humans [39].

In the second model, the pathologist will become “on the loop,” and some authors suggest we will see this in the medium term [28]. In this scenario, AI is capable of making independent decisions but pathologists are still involved. This could initially be a system whereby benign or normal tissue is screened out at an early stage leaving the pathologist to concentrate on diseased tissue. As algorithms further advance, it is possible that AI diagnoses more and more conditions, with the pathologist only needed for highly unusual or ambiguous cases. In this situation, the pathologist plays a role in quality control and oversight, checking that the decisions being made are appropriate rather than making the decisions.

In the third model, the pathologist is “out of the loop,” a scenario that some authors predict we may reach in the long term [28], at least for some decisions. It has been suggested AI could automate routine tasks [59] and more time-consuming tasks [32,60]. It is theorized that this will enable pathologists to spend more time on high-level decision-making tasks, particularly those related to disease presentations with more confounding features [60]. Tasks that have been identified in the literature for automation include those “tedious routine diagnostic tasks that require great accuracy” such as finding metastases in lymph node sections, with the potential for a significant reduction in the workload of pathologists [38]. In this situation, AI has become autonomous, and decision-making has shifted away from human control. This may be unlikely to happen in the foreseeable future, and the general consensus in the literature is that there will always be human involvement in the diagnostic process [26,28,36,37,46,53,55-57,61-65].

How pathologists might reason about and respond to these different models of working alongside AI was less clear from the literature. However, broader literature on the use of AI in health care provides some useful insights about which of these models is most likely to be responded to positively. For uptake of AI, the literature suggests that there is the need for advice in a way that recognizes the expertise of the user, making it clear that it is designed to inform and assist but not replace the clinician [66], implying that the scenario of the pathologist being “in the loop” will be most acceptable to pathologists. In thinking about this scenario, we drew on substantive theories regarding the implementation of technology and complex interventions more generally. Here, normalization process theory (NPT) provided valuable understanding. Successful introduction of technology involves interactions between individual clinicians and their work environment until the technology becomes embedded (routinely incorporated into everyday work) and integrated (sustained over time) into routine practice, a process known as “normalization” [67]. NPT suggests that, for normalization to occur, 4 key constructs need to be considered: coherence: sense-making—where individuals make sense of the new technology and how it differs from existing practice; cognitive participation: the process of engaging individuals with the introduction of the technology; collective action: how the work processes are adapted and altered to make the intervention happen; and reflexive monitoring: the formal and informal appraisal of the benefits and costs of the intervention [67-69]. This suggests that if pathologists have been able to “make sense” of AI, have been engaged in the adoption process, have been able to adapt their work processes, and are able to identify potential benefits to its introduction, it is more likely to become embedded into practice.

We also looked at theories relating to the adoption of a clinical decision support system (CDSS), with AI being a form of CDSS [66]. Relevant theories are the user acceptance and system adaptation design model and the input-process-output-engage (IPOE) model [70]. The user acceptance and system adaptation design model suggests that, for users to accept a CDSS, an iterative design process with early end-user involvement is needed, along with rigorous usability testing in both laboratory and natural settings, to ensure that the system works within the cognitive and environmental constraints of the intended user. The IPOE model suggests that acceptance of CDSS requires the CDSS to provide users with the rules that the machine followed to generate the output, so the user can make informed decisions when deciding whether to follow the recommendation.

A challenge, highlighted by NPT and the IPOE model, may be making sense of the black box of AI. In the pathology literature, it is theorized that trust is needed for uptake of AI [71]. Drawing on studies of AI implementation in other health care settings, we can also theorize that pathologists’ trust will be eroded when the AI recommendations conflict with their own observations and experience [12]. Consequently, it is theorized that the “resource” of explainable AI is necessary for building pathologists’ trust and will increase acceptance of the scenario of pathologists being “on the loop” [71].

This review has described the anticipated benefits of AI in digital pathology. While these benefits are not surprising, reflecting broader claims made for AI in the health care literature—namely, increased accuracy and efficiency—the value of this review comes from the use of a realist approach, which has allowed us to go beyond a simple listing of benefits to theorizing the contexts in which, and the mechanisms through which, such benefits are likely to be achieved. These are summarized as CMO configurations in Table 1. It has also enabled us to draw on a wider range of literature than in more traditional review approaches, an important feature given the absence of studies exploring pathologists’ attitudes toward AI [14].

We began this review with the intention of identifying stakeholders’ theories about the contextual factors that may support or constrain the adoption of AI in pathology. However, the review revealed a gap in the literature regarding the discussion of this topic, likely a reflection of the current state of progress in the development of AI in pathology. As we begin to introduce AI into pathology, there is a need for empirical research to address this gap. However, this points to an additional benefit of a realist approach, which allowed us to integrate existing theory concerning the implementation of technology and complex interventions more generally. This enabled us to develop some tentative theories regarding the mechanisms that may lead pathologists to choose to integrate AI into their work practice, providing a strong theoretical basis for future research in this area.

The review has several implications for the design and reporting of studies as we begin to move from experimental studies to real-world evaluation and the use of AI in pathology departments. Firstly, the findings highlight tasks where AI is likely to provide the greatest benefit and which benefits are most desirable in a given setting. For example, in large specialist centers, the emphasis should be on reducing workload rather than increasing accuracy. This responds to calls for developers to carefully consider the tasks that are best performed by AI and those best performed by clinicians [12]. Alternatively, the review findings can inform the selection of sites for evaluation. For example, if the ambition is to increase the accuracy of diagnosis, then smaller, nonspecialist departments are a more appropriate choice than large specialist centers. Designers also need to give careful thought to usability and how AI is integrated into the pathologist’s workflow.

The findings and resulting CMO configurations also suggest that, except for simple quantitative tasks, explainable AI will be needed for pathologists to trust the recommendations provided. However, this may be based on a misunderstanding of the current nature of explainable AI; broad descriptions of how the AI system works in a general sense can be produced but they are rarely informative with respect to individual decisions [72]. For example, in radiology, the trustworthiness of saliency maps, a widely used method to provide explainable AI in medical imaging, has been questioned [73]. Instead, rigorous evaluations should be used to provide evidence of the trustworthiness of AI, as is the case with other black-box systems in health care, such as medicines where the mechanisms of action are only partially understood [72].

Following on from this, the review findings highlight information that should be reported in evaluation studies of AI, regarding not only the size and nature of the department but also how pathologists worked in the different departments, working alone or in teams. The issue of reporting AI studies in health care has been highlighted by other authors, with recommendations for describing the study setting, the target user, the digitized workflow, and the extent of use [74-78]. Our analysis suggests that for real-world studies, a finer-grained level of reporting would be beneficial; by capturing and reporting such details, it will enable identification of the specific contexts in which AI is likely to provide the greatest benefit and therefore, where its implementation is most justified. To capture this information regarding the extent of collaborative work, as well as to capture information about the contextual factors that support or constrain the use of AI in different departments, real-world trials need to be complemented by mixed method or qualitative process evaluations.

The findings also have implications for the introduction of AI into pathology. Pathologists should be involved in the decision to introduce AI and have the opportunity to feed into evaluations of the costs and benefits of the system following its introduction.

The CMO configurations presented earlier are tentative theories, as they have not been tested with empirical data. In the next stage of this research, these will be explored and refined through interviews with pathologists. This will provide the basis for a future realist evaluation of AI in pathology, gathering empirical data to test the theories.

This paper has presented a review of stakeholders’ theories of how and in what contexts AI will be adopted and provide benefit within pathology. The results suggest that for uptake of AI in pathology, for all but the simplest quantitative tasks, measures will be required that either increase confidence in the system or provide users with an understanding of the performance of the system. For specialist centers, efforts should focus on reducing workload, rather than increasing accuracy. Designers also need to give careful thought to usability and how AI is integrated into pathologists’ workflow.