To identify relevant systematic reviews on different types of digital health education for health professionals, we used a comprehensive search strategy mentioned in Multimedia Appendix 1, including key terms for participants (eg, health professionals, health personnel, and students), intervention (eg, e-learning, patient simulation, and serious games), and article type (eg, systematic reviews and evidence synthesis). We searched the following major bibliographic databases for studies published between January 1, 2014, and July 21, 2020, without any restrictions on language or study design: MEDLINE (Ovid), Embase (Ovid), Cochrane Library, Educational Research Information Center (EBSCO), and CINAHL (EBSCO). We also manually checked the reference lists of the included systematic reviews for other potentially relevant systematic reviews on digital education for health professionals. In addition, we searched Google, Google Scholar, ResearchGate, and OpenGrey for any other studies that might be relevant for our topic, using keywords systematic review, digital education, health professionals, health professions education, eLearning, and e-learning, and reviewing either the first 10 pages or 500 results.

We included systematic reviews focusing on digital education for health care professionals in preservice (ie, student) and in-service (ie, postdegree, including postgraduate trainees and those in independent practice) positions [19]. This includes disciplines such as medicine, dentistry, nursing and midwifery, medical diagnostic and treatment technology, physiotherapy and rehabilitation, and pharmacy. Practitioners of traditional, alternative, and complementary medicine were excluded. Digital health professions education refers to health professions education that is conducted using digital technology [20] and includes modalities ranging from the basic conversion of content into a digital format (eg, a book converted into a PDF or HTML format) to more complex applications such as mobile education, digital games, virtual patients, and virtual reality (VR). Systematic reviews were included if they focused on ≥1 modality of digital education (as defined in Textbox 1) delivered in a stand-alone or blended format [20]. We defined blended education as education that incorporates aspects of traditional and digital education. Traditional education was defined as education that encompasses the use of nondigital educational materials (eg, textbooks or models) or in-person human interactions. We included systematic reviews of all the types of studies. We excluded older reviews because of the rapid evolution of the field, with the assumption that most of the active research questions from the reviews published >5 years ago would be collated in more recent reviews.

The search results from different databases were combined in a single EndNote library, and duplicate citations were removed. A total of 2 review authors (SP and BMK) screened all titles and abstracts for inclusion independently and in duplicate. Disagreements during the title and abstract screening were resolved by consensus. Full texts of articles considered eligible or uncertain based on the title and abstract screening were retrieved and screened independently and in duplicate by the same 2 authors.

From the included systematic reviews, 2 authors (SP and BMK) used a standardized form to independently extract information on the review aim; the study design, participants, interventions, comparators, and outcomes of the original research studies included; the method used to appraise the quality of the included studies (eg, risk of bias) or overall evidence (eg, the Grading of Recommendations Assessment, Development and Evaluation assessment); and recommendations for future research. We classified the outcomes according to the definitions presented in Multimedia Appendix 2. We also extracted information on all additional outcomes reported in the included reviews that did not correspond to our predefined outcome-related definitions. We classified the systematic review in terms of the single digital modality they most focused on, according to our framework (Textbox 1). In most instances, it was clear that a given review focused predominantly on 1 modality. Less often, a review encompassed multiple modalities equally, in which case we classified it as digital education; that is, the use of digital technology in health professions education in general. Finally, we identified recommendations for future research by extracting exact quotes from each review that articulated such recommendations. At every stage, disagreements between the review authors were resolved through discussion and input from the third author (LTC).

Authors SP and BMK rephrased the quoted research recommendations into research questions and then refined these by applying consistent terminology, removing duplicates, and merging or subdividing themes. Questions that focused on specific digital modalities (eg, online modules) were rewritten to make them relevant to the broader research agenda for digital health education. The final list of research questions was refined by LTC.

We developed a conceptual framework outlining various digital health professions’ education components according to the methodology described by Jabareen [18]. We consulted and built on our previous conceptual work in this area and existing frameworks for the implementation or adoption of digital education generally [30-39]. We identified these frameworks through a focused literature search on PubMed, Google, and Google Scholar. On the basis of the discussions and consensus among the review authors, key domains and subdomains were finalized. The framework and its components were represented diagrammatically. Recommendations for future research were classified according to the proposed framework in parallel by 2 authors (SP and BMK). Discrepancies were resolved through consensus and with the guidance of the third author (LTC). On the basis of the analysis of the included reviews and the observed gaps in the literature, we outline a research agenda for digital health professions education and present it in the Results section.

Of 7294 systematic reviews from our initial search, we identified 73 (1%) eligible systematic reviews (Figure 1). Another 4 systematic reviews were identified through Google, Google Scholar, ResearchGate, and OpenGrey. In total, 77 systematic reviews were included for data extraction.

The number of published systematic reviews increased over time, from 6% (5/77) published in 2014 to 56% (43/77) published in 2019. The systematic reviews focused on digital education for health professions students (17/77, 22% studies), postgraduate or independently practicing (18/77, 23% studies) health professionals, or both (42/77, 55% studies). Most of the systematic reviews were on digital education in surgery (13/77, 17% studies) or health professions education in general (ie, those not focusing on a particular type of practitioner, health care area, or topic; 36/77, 47% studies), followed by nursing (11/77, 14% studies; Table 1; Multimedia Appendix 3 [11-16,32,34,40-109]).

A breakdown of the digital modalities being investigated in the included systematic reviews is shown in Figure 2. Most systematic reviews focused on digital education in general (22/77, 29%), VR (19/77, 15%), and online education (10/77, 13%). Of the 19 reviews on VR, 17 (89%) were on VR complemented with physical objects or devices such as probes or handles and focused on psychomotor, procedural, or technical skill development. There were fewer reviews published on m-Learning (6/77, 8%), digital game-based learning (3/77, 4%), and virtual patients (2/77, 3%).

The most common outcomes in the included reviews were health professionals’ knowledge (49/77, 64%), skills (51/77, 66%), attitudes about the clinical topic (13/77, 17%), and satisfaction with digital education (18/77, 23%). Most systematic reviews compared the effectiveness of digital education to traditional education (ie, nondigital; 59/77, 77%) or other forms of digital education (35/77, 45%; Table 1). Most reviews reported only immediate, short-term outcomes; 22% (17/77) of reviews reported the impact of digital education on long-term delayed outcomes; that is, outcomes assessed with delay after the intervention [34,40-55]. Most reviews appraised methods using the Risk of Bias tool [110] only (24/77, 31%), followed by Grading of Recommendations, Assessment, Development, and Evaluation [111] (which also includes the risk of bias assessment; 22/77, 29%) and Medical Education Research Study Quality Instrument [112] (10/77, 13%). Of the 77 studies, 9 (12%) reviews did not report on the quality appraisal of the included evidence, whereas the remaining 14 (18%) reviews used different instruments to determine the evidence quality (Table 1; Multimedia Appendix 3 [11-16,32,34,40-107,109]). The included reviews mostly reported original research to be low or very low quality of evidence or reported unclear or high risk of bias in most studies (Multimedia Appendix 3 [11-16,32,34,40-107,109]). Most systematic reviews reported data from high-income countries (14/77, 18% systematic reviews) or both middle- and high-income countries (42/77, 55% systematic reviews). Only 4% (3/77) of the systematic reviews included studies from low-income countries [11,56,57]. Approximately 29% (22/77) of the included reviews did not report the setting of the included studies.

To outline different aspects of digital health professions education and identify gaps in the literature, we developed a novel conceptual framework (Figure 3) grounded in key findings of these systematic reviews together with 7 existing frameworks for digital education general [35-39,113-117] and a framework we developed previously for health professions education [33].

Broadly, the fundamental domains include an enabling and supportive context, sound infrastructure, and the optimal use of education tools and processes. The context is a combination of institutional norms, sociocultural norms, and settings in which the learner resides, as well as the level of education the learner is at. Subdomains of the context have a direct impact on the infrastructure components required and available for the delivery of digital education—digital and physical spaces, policies and regulatory standards, and human resources. Both context and infrastructure components are important in consideration of health professions education. Learners, individually and as part of a larger group, are at the core of digital health education, and their needs, preferences, prior expertise, and competencies should shape how education is delivered. The interaction among components within and across each layer is dynamic, with different parts being interconnected, as reflected using dotted lines to separate context, infrastructure, education, and learners. Studying and identifying optimal relationships between the components are handled by the research and quality assurance blocks, which are connected to the rest of the framework. Table 2 provides the detailed operational definitions for each domain of the framework.

We identified 318 discrete research questions posed in these 77 articles, from which we distilled a final list of 61 (19.2%) distinct questions covering 14 of the 16 subdomains of the above framework (Table 2; Multimedia Appendix 4 [11-16,32,34,40-95,98-107]). Research questions that spanned multiple subdomains were assigned by investigator consensus to the most relevant single subdomain. None of the included systematic reviews posed questions primarily directed at the physical infrastructure or quality assurance subdomains. We identified 26% (16/61) of questions relating to context, classified into four subdomains: sociocultural norms, institutional norms, settings, and level of education. Approximately 15% (9/61) of research questions (3 per subdomain) were identified for the digital, regulatory, and human resources subdomains within the infrastructure domain. Most of the research questions, 48% (29/61) approximately, were categorized in the education domain, which encompasses modality, instructional design, content, engagement, and evaluation subdomains. Approximately 5% (3/61) of research questions each were categorized in the learners and research domains.

We also classified the included systematic reviews based on their research questions and using our conceptual framework (Table 3; Multimedia Appendix 3 [11-16,32,34,40-107,109]). The research questions addressed by existing systematic reviews mostly revolved around digital education modality (ie, the effectiveness of various digital education modalities delivered as stand-alone or blended interventions) and content (ie, the effectiveness of digital education within a particular health care area or discipline). Some reviews assessed interactivity (engagement), various aspects of instructional design in digital education, the impact of digital education on institutional outcomes (context—institutional norms), and the impact of setting (eg, low-income and middle-income countries) on learning outcomes.

We present an evidence map of 77 systematic reviews on digital education for health professionals published between 2014 and July 2020. The reviews mostly focused on the effectiveness of various digital education modalities in surgery, health professions education in general, and nursing. Most reviews have focused on online and offline learning. Only a few reviews focused on other digital education modalities such as m-Learning, VR, digital game–based learning, and virtual patients. We developed a novel conceptual framework outlining key components of digital health professions education, namely context, infrastructure, education, learner, research, and quality assurance. Within these reviews, we identified 61 unique recommendations (questions) for future research, focusing primarily on digital education modality, instructional design, and assessment.

Our study has some limitations. First, to cover the most recent evidence in the field of digital education for health care professionals, we excluded studies before 2014; earlier reviews might have identified important research questions that remain unanswered. Our focus on systematic reviews also excluded other article types, such as editorials or viewpoints, which might have identified additional research questions. Second, although extraction and classification of research questions were done in duplicate and using a standardized approach, other classifications could be justified in some instances, which implies a degree of imprecision in the reported frequencies of specific questions. Moreover, our method did not allow us to prioritize the numerous research questions; such prioritization would require input from a representative group of experts and could be the focus of a future study. Third, there are overlaps among different concepts specified within this conceptual framework, which could be delineated and presented differently depending on potential chosen emphasis or entry points. Fourth, reviews classified as online education varied substantially in their inclusion of other modalities (eg, some expressly excluded modalities such as virtual patients, digital games, or massive open online courses, whereas others included these and other modalities). Finally, our novel conceptual framework may require revision as our understanding of this field matures and evolves, additional evidence accumulates, and new technologies emerge.

Our study also has several strengths, such as a thorough literature search for relevant studies, encompassing several indexed and gray literature databases without restrictions. We followed an established evidence map methodology and performed the steps in duplicate and independently [17]. In the development of our conceptual framework, we drew from the existing frameworks, our previous work, and discussions with experts.

We did not find other frameworks presenting a high-level overview of the use and implementation of digital education for health professionals. Therefore, we drew from the general digital education literature and found several relevant frameworks.(Multimedia Appendix 5). The included frameworks, while providing an overview of digital education, often had additional objectives such as exploring the role of specific stakeholders (eg, the private sector or the ministries), identifying barriers to adoption, or analyzing a particular digital education aspect, setting, or configuration [33,35-37,113-117]. Our framework complements other frameworks by pulling together domains previously presented only in isolation and by adding novel subdomains such as the impact of training levels, the role of regulations and accreditations, and the importance of physical infrastructure (Multimedia Appendix 4 [11-16,32,34,40-95,98-107]).

Several viewpoint articles have offered research agendas for digital health professions education [28,118-120]. They focus primarily on the design of interventions and research studies in this field, which correspond to the domains of education (modality, instruction design, assessment, and engagement), research (quality of methods and reporting), and context (setting and level of education) in our framework. The agendas espoused in these viewpoints include questions that probe more narrowly and deeply on specific issues relevant to the design and focus of future studies (eg, the choice of comparison intervention and avoidance of confounding, integration of digital education across different institutions, and the need for interdisciplinary collaboration). Our framework was intentionally broad and comprehensive and enabled us to accommodate a variety of additional questions on previously neglected topics.

Most reviews in our evidence map focused on the effectiveness of digital education interventions and rarely addressed issues around their implementation and adoption. These reviews also mostly compared the effectiveness of digital interventions with that of nondigital education. Findings from studies comparing digital and nondigital education have limited generalizability as these studies cannot account for variance within and between these 2 educational formats [118]. Future research should compare different digital education modalities as such studies are more likely to generate meaningful, generalizable findings. It should also aim to explore potential challenges related to the implementation and adoption of digital education interventions in different settings.

There is also a need for more methodologically robust research and clearer terminology in this field. The quality of the evidence, as reported in the included reviews, was relatively low, with a limited number of studies measuring skills and knowledge retention. Furthermore, it was at times difficult to determine which modality (or modalities) the included reviews focused on because of poorly explained inclusion criteria. Such ambiguity was particularly common in reviews on e-learning and blended, online, and internet-based education.

We also express concerns about the paucity of studies from low- and middle-income countries. Such countries could greatly benefit from digital education, especially by using free or low-cost education (eg, massive open online courses). Although some research findings have a universal application (eg, fundamental principles of effective learning), others (such as implementation, infrastructure, and learners) are more context specific. Given the presence of unique needs of low- and middle-income countries (eg, distinct content priorities, learner demographics, and infrastructure), we urge more research in these contexts.

Our conceptual framework will benefit researchers, funding agencies, and educators, among others. The specific questions identified and classified according to this framework provide a map for future research and can help prioritize original research studies and guide the planning of new or updated systematic reviews. We encourage investigators to broadly consider the questions we identified in this evidence map, especially those specific to areas previously less studied, such as infrastructure, learners, or quality assurance in digital education. Our framework can also be used by funding agencies to better understand the limitations of the existing research and identify areas with limited evidence with the aim of informing their funding calls in this field. Finally, this framework can encourage those developing new courses to anticipate and plan for issues that are important but might be inadvertently overlooked, such as the digital education context, infrastructure, and learners.