We followed the Cochrane methodology for every step of the review [28]. A detailed description of the methodology has been previously provided by the Digital Health Education collaboration [29]. The Digital Health Education collaboration is an international initiative evaluating the effectiveness of digital education in health professions education through a series of methodologically robust systematic reviews.

In this review, we included randomized controlled trials (RCTs) that evaluated the effectiveness of DPBL in improving health professionals’ knowledge, skills, attitudes, and satisfaction of students and compared DPBL with traditional learning methods or other forms of digital learning. Crossover trials were excluded because of a high likelihood of a carry-over effect.

We included studies with preregistration as well as postregistration health professionals as per the qualifications listed in the Health Field of Education and Training (091) of the International Standard Classification of Education [30]. However, we excluded students of alternative, traditional, and complementary medicine. We excluded studies that focused on hybrid PBL (ie, a combination of PBL and traditional learning approaches).

We included studies in which any form of digital technology was used in combination with PBL for delivering the learning content of courses, either as the sole (full digital learning) or partial (blended learning) means of delivery, for the purpose of learning in health professions education. Digital technology primarily supports PBL principles and processes by enabling contextual and collaborative learning [27]. We defined traditional PBL in line with the Maastricht model as small-group, self-directed, tutor-supported learning that revolves primarily around a problem and occurs face to face [31]. We further conceptualized the role of digital technology in PBL in line with the presented framework that builds on the Maastricht PBL framework and the Arena Blended Connected (ABC) curriculum design method [32-34]. The Maastricht PBL framework differentiates among three broad components in PBL: the first meeting, self-directed learning, and the second meeting [34]. These components are present in traditional PBL delivered face to face. Each of these three components encompasses different learning activities that we outline using the ABC curriculum design approach. The ABC curriculum design method differentiates six main learning activities that can be supported with the use of digital technology: acquisition, inquiry, practice, production, discussion, and collaboration [33]. In our framework, we present examples of how different digital tools can be employed in a variety of ways to support learning activities in PBL. For example, learning activities comprised in the first PBL component include acquisition, discussion, collaboration, practice, and investigation. Digital technology can be used in various ways to deliver each of these learning activities. For instance, acquisition of information can be achieved through the use of multimedia resources, podcasts, or text messages. Collaboration can be supported through the use of chatrooms or Web forums. Practice, on the other hand, can be facilitated via digital education modalities that support simulation such as virtual reality or virtual patients.

We analyzed studies that compared DPBL to traditional PBL or traditional learning (textbook, lectures, etc) and to different forms of DPBL interventions or other digital education.

We excluded studies that focused on individual learning interventions, evaluated the use of DPBL in other educational areas, lacked an active comparison, and assessed interventions with optional or minimal use of digital technology.

We included the following primary outcomes: (1) students’ postintervention cognitive knowledge and skills measured with any instrument (validated or nonvalidated); (2) students’ professional postintervention attitudes toward DPBL interventions, patients, or new clinical knowledge or skills measured using any instruments (validated or nonvalidated); students’ postintervention satisfaction with DPBL intervention, measured using any instrument (validated or nonvalidated).

For secondary outcomes, we focused on the economic impact of the DPBL intervention and potential adverse or unintended effect of the DPBL intervention.

We searched seven electronic databases, namely, MEDLINE (Ovid), Embase (Elsevier), Cochrane Central Register of Controlled Trials (CENTRAL, Wiley), PsycINFO (Ovid), Educational Research Information Centre (Ovid), Cumulative Index to Nursing and Allied Health Literature (Ebsco), and Web of Science Core Collection (Thomson Reuters), for relevant studies from January 1990 to August 16, 2017, without language restrictions (Multimedia Appendix 2).

We also checked the reference lists of all included studies and relevant systematic reviews and searched the International Clinical Trials Registry and metaRegister of Controlled Trials for unpublished trials. We followed the Cochrane methodology for the selection of studies, data extraction, data analysis, and risk of bias analysis, with two reviewers independently performing each of these steps [28]. From each study, the following information was extracted: first author’s surname, publication year, course name, sample size, student characteristics, intervention method, duration of study, and outcomes. We contacted the study authors for missing data and assessed the risk of bias in the included studies using the Cochrane risk-of-bias tool [28].

Standardized mean difference (SMD) for continuous outcomes, with 95% CI, was calculated based on the availability of data from the included studies. We pooled studies together based on comparison and outcomes using SMD. We interpreted the effect size using the Cohen rule of thumb (ie, with <0.2 representing no effect, 0.2 to <0.5 representing a small effect, 0.5 to <0.8 representing a moderate effect, and ≥0.8 representing a large effect) [28,35]. We employed a random-effects model in our meta-analysis. The I² statistic was employed to assess heterogeneity, with I²<25%, 25%-75%, and >75% representing a low, moderate, and high degree of inconsistency, respectively. The meta-analysis was performed using Review Manager 5.3 (Cochrane Library Software, Oxford, UK). We reported the findings in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting standards. We present the findings that we were unable to pool due to lack of data or high heterogeneity, in the form of a narrative synthesis.

The search strategy yielded 30,532 references. We included nine studies with 890 medical students (Figure 1). We excluded four studies due to missing data [20,36-38]. No relevant ongoing clinical trials were identified.

The characteristics of the nine included studies are presented in Table 1. All studies were RCTs and published in English. Seven studies were conducted in high-income countries and two studies, in middle-income countries [21,31]. Five studies had two arms, one study had three arms, and three studies had four arms. Four studies exclusively compared DPBL to traditional PBL, one study exclusively compared DPBL to traditional learning, and two studies compared DPBL to both traditional PBL and traditional learning [21,31]. None of the included studies compared different forms of DPBL. The fields of study varied across the included studies. The studies focused on pregnancy-associated urinary incontinence [39], biochemistry (acid-based physiology) [40], genetics [41], internal medicine [42], ophthalmology [21], dermatology [31], multidisciplinary [43], human physiology [22], and traumatic head injury [44].

In most studies, DPBL interventions were delivered face to face and digital technology was employed for one component of the PBL process—presentation of problems. In four studies, DPBL was at least partially delivered on a distance basis. In one study, DPBL was employed for delivery of the initial part of the PBL (ie, the first meeting; Figure 2) [43]. In three studies, DPBL was fully distance based and all components of PBL were delivered using digital technology [39,40,44]. In line with our framework, in studies with digital presentation of problems, digital technology was mostly used to support acquisition (Figure 2; Multimedia Appendix 3). In fully distance-based DPBL, digital technology enabled a range of learning activities such as acquisition, investigation, collaboration, discussion, and production. In three studies, digital technology allowed students to practice through the use of virtual reality and virtual patients [22,41,42].

The types of digital education modalities included offline learning (eg, CD-ROM) [31], online learning (eg, multimedia modules) [21,22,39,40,43,44], (immersive) virtual reality [41], and virtual patients [42] (Table 1). The control groups used traditional PBL such as text-based or paper-based PBL in six studies [22,39,40,42-44], traditional PBL or traditional learning in two studies [21,31], and solely traditional learning in one study [41]. The duration of the interventions ranged from 1 hour 33 minutes [41] to 7 weeks [31]. One study did not report the duration of the intervention [43].

Included studies reported findings on students’ knowledge, skills, attitude, and satisfaction. No studies reported cost-related outcomes or adverse/unintended effects of DPBL-based interventions. All studies measured outcomes immediately after the intervention, except Sobocan et al [42], who reported both immediate and long-term knowledge retention at the end of the academic year. The included studies mostly had an unclear risk of bias due to a lack of information on randomization, allocation concealment, and blinding of outcomes assessment (Figure 3).

The effects of DPBL compared to the traditional PBL on knowledge scores were reported in eight studies involving 822 medical students (Multimedia Appendix 4) [21,22,31,39,40,42-44]. The pooled analysis of seven studies showed little or no difference between DPBL and traditional PBL in postintervention knowledge scores (SMD 0.19, 95% CI 0.00-0.38; DPBL group, n=326; traditional PBL group, n=333; moderate quality; Figure 4) [21,22,31,39,40,42,43]. One study assessed the long-term effects on knowledge and reported no difference between the groups.

We also performed subgroup analysis based on the degree to which the digital technology was employed as part of PBL. We differentiated among studies in which digital technology was used for presentation of problems [21,22,31,42], the first part of the PBL was distance based [43], and PBL was fully distance based [39,40]. We found a statistically significant difference among these subgroups. There was a moderate improvement in postintervention knowledge scores in fully distance-based DPBL compared to traditional PBL (SMD 0.57, 95% CI 0.23-0.92) and no difference in studies on DPBL with digital presentation of problems (Figure 4).

The effects of DPBL compared to the traditional PBL on skills scores were reported in two studies (N=357). The pooled analysis of these two studies showed that DPBL may slightly improve postintervention skill scores (SMD 0.30, 95% CI 0.07-0.54; P=.01) in comparison to traditional PBL [31,43].

The effects of DPBL compared to the traditional PBL on satisfaction scores were reported in three studies with mixed findings. Two studies evaluating the use of DPBL with digitally presented problems reported no difference between DPBL and traditional PBL in satisfaction scores [21,31]. One study reported a significant difference in satisfaction scores in favor of fully digitally delivered, distance-based PBL [40]. Two studies (N=126) assessed students’ attitude toward the intervention and reported mixed results or incomplete outcome data [21,44].

The effects of DPBL compared to traditional learning on knowledge scores were reported in three studies (N=278) [21,31,41] (Multimedia Appendix 4). The pooled analysis of three studies showed that DPBL may moderately improve postintervention knowledge scores (SMD 0.67, 95% CI 0.14-1.19) in DPBL compared to traditional teaching (Figure 5). A subgroup analysis of two studies (N=210) evaluating DPBL with digital presentation of problems and traditional learning showed a large improvement in postintervention knowledge scores (SMD 0.94, 95% CI 0.56-1.31) [21,31].

The effects of DPBL compared to traditional learning on skills scores were reported in one study. This study [31] reported higher postintervention skill scores in the DPBL group than in the traditional learning group (SMD 1.13, 95% CI 0.58-1.67). The effects of DPBL compared to traditional learning on satisfaction scores were reported in two studies (N=210) with uncertain findings (SMD 0.73, 95% CI –0.17 to 1.63).

In this review, we evaluated the effectiveness of the use of digital technology for delivering PBL. Our findings show that DPBL improves students’ postintervention knowledge scores in comparison to traditional learning. DPBL is as effective as traditional PBL in improving students’ postintervention knowledge and may slightly improve postintervention skills. Moreover, fully digitally delivered, distance-based DPBL may lead to better knowledge scores in comparison to traditional PBL. The risk of bias in the included studies was mostly judged as unclear due to a lack of information on randomization, allocation concealment, and outcome assessment blinding. In the included studies, the term “blended PBL” was employed to denote diverse configurations of digital technology and PBL. For example, an intervention in which PBL was fully delivered online, but included one visit to the clinic, and another intervention in which face-to-face delivered PBL included digital presentation of problems were both termed blended learning. The use of “blended learning” therefore seemed misleading in this context. We decided to focus primarily on describing the way in which digital technology was employed in the PBL process. From our viewpoint, there are two main applications of digital technology in PBL: full or partial delivery of distance-based PBL or support of delivery of different components of face-to-face or colocated PBL. Most studies included in our review focused on colocated PBL, with digital technology used for the presentation of problems. This corresponds to the findings from two reviews focusing on the application of digital technology in PBL [23,27]. The reviews also report that in most studies, digital technology was used to provide contextual learning and present problems. One of these reviews focused on PBL from all disciplines and also highlighted the common use of digital technology for collaborative learning as part of distance-based PBL. In our review, we have found limited evidence on distance-based PBL in health professions education. There is a need for more research on this type of DPBL and to explore other applications of digital technology in face-to-face PBL, such as supporting collaboration, discussion, investigation, and practice. We present suggestions for the diverse applications of digital tools in PBL in our framework in Figure 2.

Our findings show that DPBL was more effective than traditional learning. Although we were unable to find meta-analyses comparing DPBL to other forms of education, there are numerous meta-analysis comparing traditional PBL to traditional learning [45-49]. The more recently published ones report that PBL is more effective than traditional learning, with moderate-to-large improvement in knowledge [45,46]. In our study, we found only three studies comparing DPBL and traditional learning, which reported an overall moderate improvement in postintervention knowledge scores among DPBL learners. A small subgroup analysis of studies in which digital technology was employed for the presentation of problems showed a major improvement in knowledge in the DPBL groups compared to the traditional learning groups. This may indicate that different configurations of DPBL may lead to larger knowledge gains. There is a need for more studies comparing the effect of distance-based DPBL or the use of digital technology for the support of other PBL components to the effect of traditional learning.

We found that DPBL was as effective as PBL in terms of knowledge, and fully digitally delivered distance-based DPBL was potentially more effective than traditional PBL. Although there are some nonrandomized studies on distance-based PBL corroborating this finding, the evidence from RCTs is scarce [23]. Potential reasons for the greater effectiveness of distance-based PBL include greater student interaction, involvement, and engagement among students. We also found that DPBL may be more effective than PBL for skills while satisfaction outcome data were mixed. These findings are based on a small number of studies, primarily assessing short-term effectiveness. More research is needed to evaluate the effectiveness of different configurations of digitally supported and distance-based PBL.

Our review has several limitations. Although RCTs provide the highest level of evidence for the effectiveness of an intervention, it is not always possible to use a rigorous RCT approach in educational research [50]. Studies included in this meta-analysis were designed as RCT, but most of them lacked information on the randomization method, allocation concealment, or blinding method. Furthermore, they mostly reported solely postintervention data; therefore, we could not calculate the pre-post intervention change. We assumed groups were matched at baseline for key characteristics and outcome measure scores. There was no information on DPBL in postregistration health professionals. Moreover, there was limited or no information on other outcomes such as skills, attitudes, satisfaction, costs, and adverse/untoward effects of DPBL. The included studies assessed short-term effectiveness, with only one study reporting a follow-up assessment. Strengths of our review include a comprehensive and sensitive search; clear inclusion and exclusion criteria encompassing a broad range of students, outcomes, and interventions; extraction of duplicate, independent, and reproducible data; and rigorous assessment of the risk of bias.

DPBL includes diverse applications of digital technology as part of face-to-face as well as distance-based PBL. Our findings suggest that DPBL is more effective than traditional learning and as effective as traditional PBL in improving postintervention knowledge outcomes. For improvement of skill outcomes, DPBL may be more effective than traditional learning or traditional PBL. There is limited evidence for other outcomes such as satisfaction, attitudes, cost effectiveness, and adverse effects. Most studies evaluated the use of digital technology for the presentation of problems as part of face-to-face DPBL and had unclear risk of bias. There is scope for the evaluation of digital technology in the delivery of other PBL components as well as the effectiveness of distance-based PBL.