Studies eligible for inclusion were peer-reviewed systematic reviews or meta-analyses with a clear focus on digital education for health professions education, which were published between 2020 and 2023 and written in English. The list of digital education technologies as described by Tudor Car et al [26] was used to guide eligibility screening. For convenience, definitions have been shared in Multimedia Appendix 2 (Textbox 1).

To further assess eligibility for inclusion, all the following questions had to be answered in the affirmative:

A knowledge specialist (EW) searched the following major bibliographic databases for review papers meeting the inclusion criteria and published between July 2020 and April 2023: MEDLINE (Ovid), Embase (Ovid), CINAHL (EBSCO), and the Cochrane Library. In addition, the knowledge specialist (EW) searched ERIC, Google Scholar, and Research Gate using the following search terms: systematic review, digital education, eLearning, health professionals, and health professions education. All searches were conducted on April 21, 2023.

The MEDLINE (Ovid) search strategy was identical to that used by Tudor Car et al [26] (Multimedia Appendix 3). Minor modifications were needed to convert the strategy for CINAHL (EBSCO) and the Cochrane Library.

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology was used to identify studies for inclusion. The search results were captured in RefWorks [28], and duplicates were removed. Pairs of independent reviewers screened the title and abstract of each paper against the eligibility criteria and predefined question set for inclusion. The full-text papers meeting the inclusion criteria were then retrieved and reviewed by pairs of independent reviewers. At both stages of screening, any discrepancy between reviewer pairs was resolved by a third independent reviewer.

A data capture tool was developed in Jisc Online Surveys [29] and piloted by 8 reviewers (AP, CM, EH, SK, ION, EN and 2 volunteers from the NHS England technology enhanced learning team) using a sample paper, with refinements made to the tool before data extraction from all papers included in the review.

Data extracted were study characteristics and future research recommendations as set out by the authors. Future research recommendations were mapped to the conceptual domain framework devised by Tudor Car et al [26] and any previously identified research questions, where possible. If there was insufficient alignment with existing research questions, a new one was created. All papers with future research recommendations were reviewed a second time, with any discrepancies resolved by a third independent reviewer.

AP and EW consolidated the extracted research questions by mapping them to existing questions, removing duplicates, and rephrasing those with similar meanings. New questions were then grouped and categorized by CM into overarching research questions. CM and AP discussed and reached a consensus on the final set of questions.

A 4-round modified Delphi was conducted through January to March 2024 to arrive at 5 final research questions. The method consisted of a desktop exercise to formulate the Delphi survey from the results of the umbrella review (round 1), assessment (round 2), prioritization (round 3), and ranking (round 4). Each survey was created by AP and tested by members of the research team.

The results of the umbrella review provided the material to compile a web-based survey of research questions for consideration by experts in later rounds. Research questions, previously identified in the umbrella review, meeting the inclusion criteria formed the basis of a web-based survey in round 2.

Inclusion criteria were as follows:

A total of 42 participants were recruited, including digital education professionals and simulation faculty, clinical educators, health and care workforce development and transformation leads, academics, and health care students, using the set criteria (Textbox 2). Recruitment was advertised via the Association for Learning Technologies’ weekly news digest and a nationwide digital education mailing list for health professionals maintained by NHS England (Multimedia Appendix 4).

A web-based survey, created in JISC Online Surveys and comprising all research questions from round 1, was distributed to the expert panel.

Using an adapted set of criteria from the Delphi process used by Schneider et al [30], participants were invited to anonymously score each research question against 5 criteria (Table 1).

Research questions that met the consensus threshold formed the basis for round 3.

The consensus thresholds for Round 2 were as follows:

Questions that met the inclusion criteria and those with nonconsensus were carried forward into round 3.

Participants’ voting in this round was informed by knowledge of the consensus outcomes for each research question from round 2. A web-based survey was shared with participants, consisting of 2 sections: one with research questions that met the inclusion criteria for round 2 (including the percentage consensus agreement for each question) and another with questions from round 2 that did not reach consensus.

Participants were invited to score each research question again, but this time based on priority, with 1 score assigned to each question. The scoring options were as follows: 1=not to be studied, 2=low priority, 3=medium priority, and 4=high priority.

The consensus thresholds for Round 3 were as follows:

Those meeting the inclusion criteria were taken forward for ranking in round 4.

A web-based meeting was conducted with participants, facilitated by CM and AP. Participants were sent results of the previous round in advance (Multimedia Appendix 5). The meeting offered the opportunity to discuss results from round 3 in small groups. Finally, participants were asked to select and rank their top 5 research questions independently via a web-based survey constructed in Slido [31], a web-based audience interaction tool that offers in-meeting and between-meeting polling and surveying. Participants unable to attend the meeting were invited to complete this exercise within 1 week of the meeting date. After this time, for each ranked question, points were assigned in a descending order, where higher ranks received more points, for example, a question ranked first place received 5 points, and each subsequent rank received 1 point less, down to 1 point for a question ranked fifth. The 5 research questions with the highest mean formed the final set of research priorities.

Finally, a narrative synthesis approach was chosen to summarize the diverse range of selected studies in a structured manner.

A total of 8857 potentially relevant papers were identified across all information sources. Following removal of duplicates, of the 8857 papers, 6044 (68.24%) were screened by title and abstract. Of these 6044 papers, 441 (7.3%) were assessed for eligibility via full-text review, leading to 217 (3.6%) papers being included in this review (Figure 1). Refer to Multimedia Appendix 6 for a complete list of key study characteristics for all included studies.

All included papers were systematic reviews or meta-analyses. Of the included studies across all the systematic reviews, high-income settings were predominant, with 23% (50/217) exclusively consisting of high-income countries and 33.6% (73/217) including a mix of high- and middle-income countries. In total, 14.3% (31/217) covered high-, middle-, and low-income countries, with only 0.9% (2/217) identifying middle- and low-income countries. The remaining 28.1% (61/217) of systematic reviews did not report the economic setting of their included studies.

In total, 29% (63/217) of the systematic reviews were focused on preservice alone. Of these, 11% (24/217) were nursing students, 5.1% (11/217) were medical students, and 7.8% (17/217) were mixed student populations. The least common population groups to be studied in the included reviews were pharmacy students (4/217, 1.8%), dentistry students (4/217, 1.8%), physiotherapy students (2/217, 1.8%), and health sciences students (1/217, 0.5%). This reflects the number of publications from these disciplines.

Reviews exploring the use of interventions with in-service professionals (44/217, 20.3%) are divided into fewer, less distinct categories: 13.8% (30/217) physicians, 6.5% (14/217) mixed health care professionals, and 0.5% (1/217) health care workers.

Of the systematic reviews comprising studies for both pre- and in-service professionals (110/217, 50.7%), 22.1% (48/217) focused on medical students and physicians, 17.5% (38/217) focused on mixed students and health care professionals, and 6% (14/217) focused on nursing students and nurses. Reviews comprising studies on dentistry, pharmacy, and physiotherapy were less common: 1.4% (3/217) dentistry students and dentists, 0.9% (2/217) pharmacy students and pharmacists, and 0.5% (1/217) physiotherapy students and physiotherapists.

The included reviews investigated digital education interventions for learners at a variety of stages in their career: 45.2% (98/217) were created for both students and graduates, 29.5% (64/217) for students only, 10.1% (22/217) for practicing postgraduate health care professionals, 7.8% (17/217) for a mix of trainees and postgraduate health care professionals, and 7.4% (16/217) for graduate trainees.

Figure 2 displays a chart showing the number of systematic reviews published for each digital modality, organized by publication year. Several additional digital modalities were identified in the systematic reviews over and above the modalities included in the study by Tudor Car et al [26]. These were consolidated into three new categories:

Refer to Multimedia Appendix 2 for a full list of definitions.

Most systematic reviews focused on XR and immersive technologies (132/217, 60.8%) and online digital education (99/217, 45.6%). The least focus was given to serious gaming and gamification (16/217, 7.4%), simulation (15/217, 6.9%), massive online open courses (7/217, 3.2%), and AI (6/217, 2.8%).

Publication numbers were highest in 2022, although the annual totals between January 2020 and April 2023 (Figure 2) exceed those published in the original study by Tudor Car et al [26].

Mapped against the conceptual framework of digital health education for health care professionals [26], most systematic reviews were focused on education modality (170/217, 78.3%) and instructional design (99/217, 45.6%). Very few reviews focused on human resource requirements (7/217, 3.2%) and regulatory factors (6/217, 2.8%) in relation to digital education infrastructure, with only 0.9% (2/217) of reviews concerned with quality assurance of digital health education in the setting.

The majority of studies emphasized reporting educational outcomes related to skills (161/571, 28.1%) and knowledge (133/571, 23.3%), followed by studies focusing on satisfaction (77/571, 13.5%), attitude (54/571, 9.5%), and behavior (38/571, 6.7%). Few studies (29/571, 5.1%) reported patient outcomes.

A total of 151 future research recommendations were extracted from the 217 systematic reviews. These were analyzed, recategorized, and consolidated by 3 authors (AP, CM, and EW) to create a final list of 63 updated research questions. Refer to Tables 2 and 3 for the complete lists.

Across the questions, online digital education (299/1257, 23.7%) and XR and immersive technologies (266/1257, 21.1%) were the most predominant digital modalities requiring further research. Most questions suggest further inquiry within the following domains of education: modality (21 questions), instructional design (16 questions), and assessment (14 questions).

The least predominant modalities are massive online open courses (20/1257, 1.59%) and AI (18/1257, 1.43%).

The evidence behind this research question points to several related yet distinct avenues of inquiry. A common theme among them is the need for further research to demonstrate how learning outcomes translate into real-world contexts, both for technical and nontechnical skills, particularly in nursing and medical fields [42-44]. One study specifically recommends further research to ensure that the competence and confidence in clinical practice skills demonstrated in a learning environment are maintained when applied in the clinical setting, where additional external factors may come into play [44]. Ensuring that the learning experience is relevant and closely mirrors real-world situations is a key consideration when designing instruction to facilitate effective transfer of knowledge and skills [248].

Furthermore, studies should track and evaluate the effectiveness of learning throughout a health care professional’s career [41], using clinical outcomes and patient-reported outcome measures to assess the transfer of learning [45].

The use of AI to optimize health education is a rapidly developing field. Current evidence proposes further research on the potential of this technology to elevate and shift performance assessment from benchtop and simulated environments to real-world context, offering more precise and scalable feedback mechanisms for both trainees and practitioners. For instance, machine learning has already shown promise in surgical skills training, where it could provide automated, objective assessments of technical skills, provided that more reliable assessment data were available [57]. Besides enhancing existing approaches to learning and assessment, AI also enables new modalities, such as personalized learning, which can enable more self-directed learning and may also support learner motivation by connecting data-driven insights to learning experiences, and adaptive learning [249]. AI integrated into personalized or adaptive learning could help students practice clinical decision-making by simulating different patient cases customized to their knowledge gaps based on previous performance. Furthermore, AI could generate dialogue, actions, and complex situations that replicate clinical settings, further enabling students to develop critical skills with reduced risk [56]. However, while AI offers substantial benefits, its implementation must be carefully considered to mitigate against potential challenges such as ethical concerns, bias from training data, and over-reliance on technology [249]. To safely and effectively adopt AI into health care education, a careful approach could integrate AI in a way that complements existing strategies that emphasize critical thinking and adaptability as well as technical proficiency.

Across the spectrum of digital education modalities, several studies highlighted the need for research to adopt appropriate study designs to determine learning gains [35, 63, 80, 87, 102,123,128,129,179,199,203,226]. Designs need to be larger in scale [39,60,105,107,113,124,134,140,188,214,218,246], multicentered [45,134,227,245], and carry out longer-term follow-up [37,62,81,105,107,128,133,140,149,195,223,241] to ensure retention and transfer of learning. Many of the included studies recommended more randomized controlled trials [37,38,62,81,100,105,115,124,140,188,195,214,222,236,241], although one paper supports the suggestion provided by Tudor Car et al [211] that quasi-experimental designs may be better suited to educational research [144] and another study, in relation to cultural competency, suggests that observational studies and other designs may be preferred [52].

Key recommendations from the data sources that derive this research question are primarily concerned with the integration and sequencing of pedagogical devices (ranging from flipped classrooms, timing of simulation briefing, e-learning, and active vs didactic approaches) in health education training and curricula as well as the quality of the approach used [41,42,82,95,102,119,133,142]. Some studies suggest delving further by closely comparing and evaluating the impact of nuanced difference in the design elements of an intervention or specific educational technology [37,63,82,101,130].

Stepping back from this detail within the intervention design itself, studies also reinforce the need to define educational needs, the context, and the setting to select suitable, appropriate, and effective modalities for specific learner groups [40,80,103,126,232].

Looking to the future, some studies highlight several novel technologies and suggest exploring their potential for designing more advanced educational interventions, such as AI [56], adaptive learning [87], wearable technology [213], eye tracking for assessment [55,61], and gamification to increase learner motivation [61].

While many still recommend further comparison studies between digital and nondigital interventions [33,48,119], others support the recommendation by Tudor Car et al [26] that comparisons between digital-to-digital interventions are now needed and necessary [142,231].

Principally, the foundations of this research question lie within a body of evidence that urges a more standardized approach to outcome measurement, using validated measurement tools [81,98,120,141,165]. Beyond this, there is little guidance on how to achieve this, although 1 study (specific to surgical skills training) suggests that machine learning could help identify which metrics accurately assess skills [201].

In many ways, each of these research priorities are connected and interdependent. For example, to be able to measure learning transfer, learning outcomes need to be defined from the beginning and integrated into the learning design. This, in turn, will influence the research methodology used.