This review is a part of a global evidence synthesis initiative on digital health professions education for which a wider search strategy was developed (Multimedia Appendix 1). The following databases were searched from January 1990 to August 2017: Medical Literature Analysis and Retrieval System Online (Ovid), Excerpta Medica dataBASE (Ovid), Central Register of Controlled Trials (CENTRAL; Cochrane Library), PsycINFO (Ovid), Education Resources Information Center (Ovid), Cumulative Index to Nursing and Allied Health Literature (EBSCO), and Web of Science Core Collection (Thomson Reuters). The search was limited to studies reported after 1990, as before this, the use of computers was limited to very basic tasks.

No language or publication restrictions were applied. We searched reference lists of all included studies and relevant systematic reviews. The International Clinical Trials Registry Platform Search Portal and Current Controlled Trials metaRegister of Controlled Trials were also searched to identify unpublished or ongoing trials, as well as meeting abstracts and PhD theses. We contacted study authors of included studies to ask if they were aware of other relevant studies and to provide full reports where these were not identified by the searches. Search results from different sources were combined in a single library and duplicate records were removed.

We included individual or cluster randomized trials (cRCTs) that compared digital education to self and usual or traditional learning or other forms of digital education to train pre or postregistration health professionals to deliver smoking cessation therapy. Health professionals with qualifications listed in the Health Field of Education and Training (091) in the International Standard Classification of Education [20] were included. We excluded studies of students and/or practitioners of traditional, alternative, and complementary medicine. Digital education interventions could be delivered as the main mode of the education intervention or as a part of a complex, multicomponent intervention (ie, blended education). We accepted any type of digital education such as offline and Web-based, computer-based digital education, Serious Gaming and Gamification, massive open online courses, virtual reality environments, virtual patient simulations, psychomotor skill trainers, and mobile learning or mLearning [21-25]. No restrictions on outcomes were applied.

Moreover, 2 reviewers (MS and SB) independently screened titles and abstracts identified by the searches. Full texts of potentially relevant articles were obtained and independently assessed for inclusion by 2 reviewers (MS and SB). Where data were missing or incomplete, authors were contacted for additional information [26,27]. Any disagreements were resolved through discussions between the 2 reviewers with a third reviewer acting as an arbiter (RB).

A total of 2 reviewers (MS and SB) independently extracted data using a standardized data extraction form, which was piloted and amended on the basis of feedback. Data were extracted on study design, participants’ demographics, type of digital education, intervention content, and outcomes. We contacted study authors of the included studies in case of any unclear or missing information. Disagreements between reviewers were resolved by discussion. A third reviewer (RB) acted as an arbiter in cases where disagreements persisted.

Data on the following primary outcomes were extracted:

We also extracted data on the following secondary outcomes:

For continuous outcomes, we extracted data to calculate standardized mean difference (SMD) and associated 95% CIs in change from baseline or at follow-up between intervention and control groups. For studies that reported median and range for the various outcomes, we converted this to mean and standard deviation [28]. For dichotomous outcomes, we extracted data to calculate relative risks (RRs) and 95% CIs. Where studies reported more than 1 measure for each outcome, the primary measure as defined by the study authors was extracted.

The methodological quality of included randomized controlled trials was independently assessed by 2 reviewers (MS and SB) using the Cochrane risk of bias tool, which includes the following domains: random sequence generation, allocation concealment, blinding of participants to the intervention, blinding of outcome assessment, attrition, and selective reporting. We also assessed the additional domain of baseline imbalances [17]. The following additional criteria were included for the assessment of cRCTs: recruitment bias that can occur when individuals are recruited to the trial after the clusters have been randomized, loss of clusters, incorrect analysis, and comparability with individually randomized trials [17].

We grouped studies according to outcomes assessed—skill, knowledge, attitude, satisfaction, practice and behavior change, self-efficacy, patient outcomes, and cost. Within these outcomes, we further grouped studies on the basis of intervention (digital education or blended education) and comparison (usual learning or traditional education, blended education, or other forms of digital education)

Heterogeneity was assessed visually using forest plots and by considering differences in participants, interventions, and outcomes across studies. Due to substantial differences among studies, we used a narrative approach to data synthesis. We were unable to identify a clinically meaningful interpretation of effect size in the literature for digital education interventions. Therefore, in line with other research in the field, we present outcomes using postintervention SMD and interpret the effect size using Cohen rule of thumb (ie, with 0.2 representing a small effect, 0.5 a moderate effect, and 0.8 a large effect) [17]. For dichotomous outcomes, we summarized RRs and associated 95% CIs across studies. Subgroup analyses were not feasible because of the small number of studies.

A total of 3 studies were judged as high risk of bias for at least 1 domain; all other studies were rated as unclear risk of bias. Concealment of treatment allocation and blinding of outcome assessors were particularly, poorly reported with only 2 studies judged as low risk of bias for each of these domains. Information on incomplete outcome data and selective outcome reporting was better reported with 2 studies judged as unclear risk of bias for incomplete outcome data and 1 for selective outcome reporting. Furthermore, 2 studies were judged as high risk of bias for incomplete outcome reporting and 1 for selective outcome reporting. No other domains were judged high risk of bias for any of the included trials (Figure 2). For cRCTs, none of the studies were judged as high risk of bias for any domain; all studies were judged as either low or unclear risk of bias for recruitment, loss of clusters, and incorrect analysis.

A total of 5 studies (313 participants) assessed postintervention knowledge gain using multiple-choice questions (MCQs) [26,30,35,37] or questionnaires [36] (Multimedia Appendix 2). None of the studies used validated instruments to measure knowledge. Moreover, 3 studies provided quantitative data and found that knowledge gain was similar with digital education compared with usual learning and for blended education compared with digital education (Figure 3). Furthermore, 2 studies did not provide numerical data. One study compared 2 different types of digital education (Web-based multimedia training compared with a website with hyperlinks to downloadable reading material) and found no difference between the interventions [35]. The other study compared blended education with digital education and found no difference in postintervention knowledge between groups [36].

A total of 5 studies (3293 participants) assessed postintervention skill to deliver smoking cessation therapy using objective structured clinical examination scores, [29,33,37] a Likert scale [26], or practitioner checklist [34] (Multimedia Appendix 2). Only 1 study used a validated assessment tool [33]. Furthermore, 1 study found that blended education was associated with greater improvement in skill for delivering smoking cessation advice compared with digital education alone (SMD=0.58, 95% CI 0.08-1.08) [37]. A second study found greater improvement in skill with blended education compared with usual learning (RR=2.04, 95% CI 1.51-2.76) [29]. However, a further study that compared blended education with usual learning found no difference between groups (SMD=−0.05, 95% CI −0.15 to 0.05) [33]. A study that compared 2 types of digital education involving a computer-based program with or without a performance specific feedback system found no difference in skill between groups (RR=1.01, 95% CI 0.96-1.08) [34].

Another study reported small improvements in postintervention skill of health professionals compared with usual learning; however, no numerical data were reported for quantitative analysis [26].

A total of 4 studies (532 participants) assessed postintervention attitude toward educational interventions [30,32] and new knowledge [31,36] using Likert scales [30-32] and a questionnaire [36] (Multimedia Appendix 2). None of the studies used a validated instrument.

Moreover, 1 study reported improved attitudes following digital education compared with usual learning (SMD=0.45; 95% CI 0.18-0.72) [31]. Furthermore, 3 studies did not provide quantitative data. In addition, 2 of these compared digital education to usual learning and reported positive attitudes toward digital education [30,32]. Another study reported no difference in postintervention attitude between blended education and digital education [36].

A total of 3 studies (415 participants) assessed postintervention satisfaction with the educational interventions using nonvalidated MCQs [37] and Likert scales [31,35] (Multimedia Appendix 2). Moreover, 1 study reported greater satisfaction after blended education compared with digital education (SMD=0.62, 95% CI 0.12-1.12) [37]. In addition, 2 studies did not provide any quantitative outcome data. Out of these, 1 reported higher postintervention satisfaction in digital education compared with usual learning [31]. Another study reported that postintervention satisfaction was higher with Web-based digital education compared with digital education through websites with hyperlinks [35].

A total of 4 studies (650 participants) assessed postintervention practice and behavior change using nonvalidated surveys [29] and Likert scales [26,31,35,36] (Multimedia Appendix 2). Furthermore, 1 study reported higher postintervention readiness to change practice and behavior to help patients quit smoking with blended education compared with the usual learning (RR=1.58, 95% CI 1.25-2.00) [29]. Another study reported no difference in tobacco cessation–related behaviors between the digital education and usual learning groups (SMD=0.13, 95% CI −0.14 to 0.40) [31]. Furthermore, 1 study reported no difference in postintervention practice and behavior change between 2 modalities of digital education [35]. Similarly, no difference was observed between digital education compared with usual learning in another study [26,36]. No numerical data were reported for these 2 studies to be included in the quantitative analysis.

A total of 6 studies (1988 participants) assessed postintervention self-efficacy using nonvalidated questionnaires [30,37], Likert scales [26,31,35], and a 6 item checklist [33] (Multimedia Appendix 2). Moreover, 1 study reported no difference in postintervention self-rated smoking cessation counseling skill between blended education and digital education (RR=0.38, 95% CI 0.12-1.13) [37]. Another study reported no difference in self-efficacy scores toward providing tobacco cessation interventions in the digital education and usual learning group (SMD=0.17, 95% CI web 0.10 to 0.44) [31]. In 1 study, higher number of participants receiving blended education reported self-efficacy for performing smoking cessation-related counseling compared with usual learning (P<.05) [33]. In another study, change in self-efficacy between baseline and postintervention was significantly greater with digital education compared with usual learning (P=.03) [26]. Moreover, 1 study reported no difference in postintervention self-efficacy between 2 modalities of digital education [35]. In another study, where self-efficacy was measured only in the intervention group, computer-based program was reported to be effective in improving smoking cessation counseling skill by 73.34% of intervention group participants [30]. No numerical data were reported for these 4 studies to be included in the quantitative analysis [29,31,35,37].

There was 1 study which assessed postintervention patient outcomes such as smoking index, general health score, quality of life score, and the cost associated with the blended education intervention (Multimedia Appendix 2). The total cost including intervention and health care cost per practice was estimated to be US $2384. However, no quantitative data were reported [27].

Our review included 11 studies that investigated the effectiveness of digital education for training health professionals to deliver smoking cessation therapy. No difference was found between digital education and traditional or usual learning. There was some suggestion that blended education results in greater improvements in satisfaction, skill, and knowledge compared with digital education alone. There was also some evidence for improved attitude following digital education compared with usual learning. Only 1 study reported patient outcomes and the setup cost of digital education. There were insufficient data to investigate what components of the digital education interventions were associated with the greatest improvements in learning outcomes. Studies were poorly reported, heterogeneous, assessed a broad range of different outcomes, and compared different types of interventions on the range of pre and postregistration health professionals. The findings of this review should therefore be interpreted with some caution.

As far as we are aware, this is the first review to address the topic of digital education to train health professionals to deliver smoking cessation therapy. We followed best practice methods for systematic reviews, which attempted to minimize risk of bias and errors in the review process. We conducted a comprehensive sensitive search across a broad range of databases and included additional steps to identify unpublished studies such as searching trials registers, meeting abstracts and PhD theses, screened references of included studies, and contacted authors of abstracts for further information. It was not possible to formally assess the risk of publication bias because of the small number of heterogeneous studies included in our review, but given our extensive search, we consider it unlikely that relevant studies have been missed. Moreover, 2 independent reviewers were involved in all stages of the review process, standardized data extraction forms were used, and we used an accepted tool to assess the risk of bias in the included studies. This identified potential limitations in the included studies, particularly in reporting, which meant that many of the risk of bias domains were judged as unclear for the majority of studies. The included studies evaluated a broad range of interventions and outcomes; therefore, it was not appropriate to calculate summary effect estimates. The included digital education interventions mostly comprised asynchronous Web-based programs aimed at postregistration health professionals. For busy health professionals, digital education is a convenient avenue for fulfilling continuing medical education requirements and promoting knowledge and skill in particular areas in which they may not have previously had training in. Furthermore, 2 studies included an interactive or feedback component in the digital education intervention [34,36]. Evidence suggests that interventions with feedback and interactivity can enhance engagement and consequently the effectiveness of learning [38,39]. However, there were insufficient data available in our review to perform subgroup analyses or a more advanced statistical analysis to evaluate what components of the included interventions contributed to the greatest improvement in outcomes.

There were a number of limitations with the included studies. Reporting of the digital education interventions, especially the description of the intervention, aims and outcomes, pedagogical approach, and use of validated outcome assessment instruments was inconsistent across the studies. There was also a lack of baseline assessment in some studies, meaning that only postintervention data could be used in the analysis, potentially biasing results. Several studies did not report numerical data; therefore, they could not be formally included in our synthesis. We have included results available from these studies, but had additional data been available in the included studies, these would have allowed us to conduct a more informative analysis.

When considering the implications of the findings of our review for practice, it is important to consider the implications of the differences in the effectiveness of different types of education. For forms of digital education that have additional benefits compared with standard education (eg, ability to target larger numbers of people, self-paced learning), showing that these are as effective as standard education is likely to be sufficient to recommend the use of these types of education. In contrast, for types of digital education where technology is supplementing standard face-to-face learning (eg, blended education or use of “clickers”), outcomes would need to be better with digital education than with standard education. For example, the study evaluating the use of clickers to give instructions during classroom lectures reported a positive impact of technology on learners’ attitudes, which consequently improved learning outcomes [32]. Our findings suggest that computer-based education, both Web-based and offline, is at least as effective as usual or traditional learning for smoking cessation therapy training in health professionals. This suggests that computer-based learning is an appropriate method to deliver training for health professionals. We found that blended education appears to offer additional benefits compared with digital education or traditional education alone. However, given the additional costs of this type of education, further studies on the cost effectiveness of blended education are needed before this can be recommended for use in practice. There were insufficient data from the studies included in our review to make specific recommendations regarding what types of digital education or components of digital education are likely to be most effective.

There is a need for further robust studies in a range of settings to determine the true potential of digital education to train health professionals to deliver smoking cessation therapy. More studies evaluating patient outcomes such as postintervention smoking cessation rates, smoking status, and abstinence are needed to assess the effectiveness of digital education for health professionals. Before recommending implementation of digital education programs, information on cost effectiveness, sustainability, as well as the direct and indirect costs such as time to develop as well as implement a smoking cessation module is needed. This will help policy makers to make practical recommendations and allocate resources appropriately. More research is needed to understand the feasibility of integrating digital smoking cessation training methods into the curriculum and continued medical education; research is also needed to understand the short-term and long-term effects in different geographical, socioeconomic, and cultural settings. In addition, sustainability, cost savings, and accreditation of digital smoking cessation therapies in health professional training need to be further researched.

Many digital education interventions in the included studies were based on smoking cessation guidelines. However, validation of the course content and use of underpinning learning theories to develop the pedagogy were lacking in most of the studies. Largely, the studies focused on integrating the new technology into the existing curriculum as opposed to using learning theories to design the digital education intervention for successful delivery of education. An increasing body of evidence reveals that theory-based interventions have greater impact than those that are not based on theory. Appropriate use of learning theory and pedagogy framework along with sound methodology can enable more robust studies to be conducted on digital education with research questions adequately addressed through theoretically informed research design, data collection, and analysis [40-42]. Future studies should therefore focus on developing, delivering, and evaluating digital education with a strong pedagogical foundation.

Digital education can contribute significantly to the World Health Organization (WHO) mission to transform and scale up health professionals’ education by filling the medical education divide between low- and high-income countries [43]. Given the greater prevalence of smoking in low-income countries and lesser awareness of harms of smoking compared with high-income countries, being able to train health professionals to deliver smoking cessation advice is of particular importance for low-income countries. However, none of the studies included in our review assessed the use of digital education in such resource-constraint settings. This is an important area for future research.

We did not identify any studies on advanced educational technologies for digital education such as mobile learning, virtual patients, virtual reality environments, or serious gaming, which have the potential to transform education for health professionals. The lack of smoking cessation studies evaluating these educational technologies makes it difficult to make recommendations for integrating such pivotal digital technologies into health professional education. In the future, it is important to have a more detailed reporting of different components of digital education interventions to allow for a more thorough analysis of the most active and effective components.

Digital education appears to be at least as effective as usual or traditional learning in improving health professionals’ knowledge and skill for delivering smoking cessation therapy. This suggests that digital education is an appropriate method to deliver training for health professionals on how to deliver smoking cessation therapy. However, limitations in the evidence base mean that these conclusions should be interpreted with some caution. There was insufficient evidence to determine what components of digital education are associated with the greatest improvements in outcomes, although there was some evidence that blended education may be more effective than either digital education or usual learning alone.