This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Tobacco smoking, one of the leading causes of preventable death and disease, is associated with 7 million deaths every year. This is estimated to rise to more than 8 million deaths per year by 2030, with 80% occurring in low- and middle-income countries. Digital education, teaching, and learning using digital technologies have the potential to increase educational opportunities, supplement teaching activities, and decrease distance barriers in health professions education.
The primary objective of this systematic review was to evaluate the effectiveness of digital education compared with various controls in improving learners’ knowledge, skills, attitudes, and satisfaction to deliver smoking cessation therapy. The secondary objectives were to assess patient-related outcomes, change in health professionals’ practice or behavior, self-efficacy or self-rated competence of health professionals in delivering smoking cessation therapy, and cost-effectiveness of the interventions.
We searched 7 electronic databases and 2 trial registers for randomized controlled trials published between January 1990 and August 2017. We used gold standard Cochrane methods to select and extract data and appraise eligible studies.
A total of 11 studies (number of participants, n=2684) were included in the review. All studies found that digital education was at least as effective as traditional or usual learning. There was some suggestion that blended education results in similar or greater improvements in knowledge (standardized mean difference, SMD=0.19, 95% CI −0.35 to 0.72), skill (SMD=0.58, 95% CI 0.08-1.08), and satisfaction (SMD=0.62, 95% CI 0.12-1.12) compared with digital education or usual learning alone. There was also some evidence for improved attitude (SMD=0.45, 95% CI 0.18-0.72) following digital education compared with usual learning. Only 1 study reported patient outcomes and the setup cost of blended education but did not compare outcomes among groups. There were insufficient data to investigate what components of the digital education interventions were associated with the greatest improvements in learning outcomes.
The evidence suggests that digital education is at least as effective as usual learning in improving health professionals’ knowledge and skill for delivering smoking cessation therapy. However, limitations in the evidence base mean that these conclusions should be interpreted with some caution.
PROSPERO CRD42016046815; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=46815
Tobacco smoking, one of the leading causes of preventable death and disease, is associated with 7 million deaths every year. This is estimated to rise to more than 8 million deaths per year by 2030, with 80% occurring in low- and middle-income countries [
Digital education, teaching, and learning using digital technologies have the potential to increase educational opportunities, supplement teaching activities, and decrease distance barriers in health professional education [
Previous systematic reviews that have evaluated digital education for smoking cessation and prevention have focused on interventions aimed at patients rather than training health professionals [
If digital education is at least as effective as standard face-to-face learning methods, then there is the potential for digital education to be used to deliver training in smoking cessation therapy with associated benefits such as being able to reach a much larger audience and allowing more flexibility in when and where the training is undertaken. The evidence could be used to make recommendations regarding the optimal digital education approach to train health professionals to deliver smoking cessation therapy.
The Cochrane recommendations for the conduct of systematic review were followed, and this review is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidance [
This review is a part of a global evidence synthesis initiative on digital health professions education for which a wider search strategy was developed (
No language or publication restrictions were applied. We searched reference lists of all included studies and relevant systematic reviews. The International Clinical Trials Registry Platform Search Portal and Current Controlled Trials metaRegister of Controlled Trials were also searched to identify unpublished or ongoing trials, as well as meeting abstracts and PhD theses. We contacted study authors of included studies to ask if they were aware of other relevant studies and to provide full reports where these were not identified by the searches. Search results from different sources were combined in a single library and duplicate records were removed.
We included individual or cluster randomized trials (cRCTs) that compared digital education to self and usual or traditional learning or other forms of digital education to train pre or postregistration health professionals to deliver smoking cessation therapy. Health professionals with qualifications listed in the Health Field of Education and Training (091) in the International Standard Classification of Education [
Moreover, 2 reviewers (MS and SB) independently screened titles and abstracts identified by the searches. Full texts of potentially relevant articles were obtained and independently assessed for inclusion by 2 reviewers (MS and SB). Where data were missing or incomplete, authors were contacted for additional information [
A total of 2 reviewers (MS and SB) independently extracted data using a standardized data extraction form, which was piloted and amended on the basis of feedback. Data were extracted on study design, participants’ demographics, type of digital education, intervention content, and outcomes. We contacted study authors of the included studies in case of any unclear or missing information. Disagreements between reviewers were resolved by discussion. A third reviewer (RB) acted as an arbiter in cases where disagreements persisted.
Data on the following primary outcomes were extracted:
Learners’ knowledge postintervention. Knowledge is defined as the learners’ factual or conceptual understanding measured using difference in pre and posttest scores.
Learners’ skill postintervention. Skill is defined as the learners’ ability to demonstrate a procedure or technique in an educational setting.
Learners’ attitudes postintervention toward digital education or toward new clinical knowledge and skill or patients (eg, awareness of moral and ethical responsibilities involved in patient contact). Attitude is defined as the tendency to respond positively or negatively toward the intervention.
Learners’ satisfaction postintervention with the learning intervention (eg, retention rates, dropout rates, and survey satisfaction scores). This is defined as the level of approval when comparing the perceived performance of digital education with one’s expectations.
We also extracted data on the following secondary outcomes:
Patient-related outcomes (eg, heaviness of smoking index, number of patients who are stopping smoking).
Change in health professionals’ practice or behavior.
Self-efficacy referred to as self-rated competence of health professionals in delivering smoking cessation therapy.
Cost and cost effectiveness of the intervention.
For continuous outcomes, we extracted data to calculate standardized mean difference (SMD) and associated 95% CIs in change from baseline or at follow-up between intervention and control groups. For studies that reported median and range for the various outcomes, we converted this to mean and standard deviation [
The methodological quality of included randomized controlled trials was independently assessed by 2 reviewers (MS and SB) using the Cochrane risk of bias tool, which includes the following domains: random sequence generation, allocation concealment, blinding of participants to the intervention, blinding of outcome assessment, attrition, and selective reporting. We also assessed the additional domain of baseline imbalances [
We grouped studies according to outcomes assessed—skill, knowledge, attitude, satisfaction, practice and behavior change, self-efficacy, patient outcomes, and cost. Within these outcomes, we further grouped studies on the basis of intervention (digital education or blended education) and comparison (usual learning or traditional education, blended education, or other forms of digital education)
Heterogeneity was assessed visually using forest plots and by considering differences in participants, interventions, and outcomes across studies. Due to substantial differences among studies, we used a narrative approach to data synthesis. We were unable to identify a clinically meaningful interpretation of effect size in the literature for digital education interventions. Therefore, in line with other research in the field, we present outcomes using postintervention SMD and interpret the effect size using Cohen
Our search strategy for a series of systematic reviews focusing on different digital health professional education modalities yielded 30,532 unique references. Upon screening of titles and abstracts, we excluded 30,051 ineligible references and retrieved full texts for 22 potentially eligible studies. We excluded 10 studies that did not meet the inclusion criteria: 3 were not randomized trials, 4 did not evaluate digital education intervention, and 3 did not target health professionals. A total of 11 studies (12 reports; 2684 health professionals) were included in the review—8 individually randomized trials and 3 cluster randomized trials. Furthermore, 1 study was reported in 2 separate journal articles [
All included studies were published in English. In addition, 7 studies focused on postregistration health professionals—3 were restricted to doctors [
Several modalities were used to deliver the digital education intervention. Web-based systems were used in 6 studies where participants could access learning materials through a Web gateway [
Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram. RCT: randomized controlled trial.
Characteristics of included studies.
Study, country, design | Participants (N) | Participants details | Intervention | Control | Learning outcomes | ||||||
Chaikoolvatana 2009, Thailand (RCTa) [ |
85 | Pharmacy students | Interactive computer program for smoking cessation counseling | Classroom lectures | Knowledge and attitude toward intervention | ||||||
Gordon 2013, United States of America (RCT) [ |
215 | Respiratory therapists, nurses, and nurse practitioners | Web-based smoking cessation education program | Usual traditional learning | Behavior, attitude, and self-rated efficacy in providing smoking cessation therapy | ||||||
Young 2002, Australia (RCT) [ |
53 | Family physicians | Web-based distance learning module for delivering smoking cessation advice | Preventive care guidelines sent via postal mail | Knowledge, skill, readiness to change, and self-rated competence | ||||||
Galal 2015, United States of America (RCT) [ |
214 | Pharmacy students | Use of student response systems (SRS) or “clickers” for instruction in a smoking cessation module | Instruction without student response systems | Learner’s attitude toward intervention | ||||||
Butler 2013, United Kingdom (cRCTb) [ |
53 | General practitioners, and nurses | Web-based learning program with face-to-face trainings to deliver behavior change counseling in smoking cessation | Usual traditional learning | Patient reported changes in smoking behavior after health professionals training | ||||||
Hymowitz 2007, United States of America (cRCT) [ |
16 | Pediatric residents | Hybrid CD-ROM/website training program and seminars series to deliver smoking cessation therapy | Usual learning with reading material on smoking cessation | Skill, practice, and behavior change | ||||||
Ockene 2015, United States of America (cRCT) [ |
1503 | Medical students | Web-based multi-modal education and face-to-face trainings for smoking cessation counseling | Usual learning with traditional tobacco education in the medical curricula | Smoking cessation counseling skill and self-rated competence | ||||||
Bonevski 1999, Australia (RCT) [ |
19 | General practitioners | Computer-based program with performance specific feedback system for screening smoking behavior | Computer program without feedback system for smoking cessation behavior | Screening smoking behavior (smoking status classification) | ||||||
Stoner 2014, United States of America (RCT) [ |
92 | Physicians, nurse practitioner, and physician assistants | Web-based multimedia training program for screening, brief intervention, and referral to treatment | Website with hyperlinks to downloadable reading materials | Knowledge, satisfaction, self-efficacy, and change in clinical practice | ||||||
Brunette 2015, United States of America (RCT) [ |
46 | Psychiatrists, advanced nurse practitioners | Videoconference educational outreach and use of printed slides, handouts, and questionnaires for cessation pharmacotherapy | Interactive in-person lecture with slides and handouts | Knowledge and attitude toward intervention | ||||||
Stolz 2012, Switzerland (RCT) [ |
129 | Medical students | Self-directed Web-based module | Lectures with video demonstration on smoking cessation | Knowledge, skill, satisfaction, and self-rated smoking cessation counseling skill |
aRCT: randomized controlled trial.
bcRCT: cluster randomized controlled trial.
Comparison interventions also varied across the studies. A total of 7 studies included a traditional learning control group: face-to-face workshops or lectures in 3 studies [
A total of 3 studies were judged as high risk of bias for at least 1 domain; all other studies were rated as unclear risk of bias. Concealment of treatment allocation and blinding of outcome assessors were particularly, poorly reported with only 2 studies judged as low risk of bias for each of these domains. Information on incomplete outcome data and selective outcome reporting was better reported with 2 studies judged as unclear risk of bias for incomplete outcome data and 1 for selective outcome reporting. Furthermore, 2 studies were judged as high risk of bias for incomplete outcome reporting and 1 for selective outcome reporting. No other domains were judged high risk of bias for any of the included trials (
A total of 5 studies (313 participants) assessed postintervention knowledge gain using multiple-choice questions (MCQs) [
A total of 5 studies (3293 participants) assessed postintervention skill to deliver smoking cessation therapy using objective structured clinical examination scores, [
Risk of bias summary: reviewers' judgements about each risk of bias item for each included study (RCTs and cRCTs).
Forest-plot of outcomes showing standardized mean deviations (SMDs) and 95% CI for studies comparing digital education or blended education with usual learning.
Another study reported small improvements in postintervention skill of health professionals compared with usual learning; however, no numerical data were reported for quantitative analysis [
A total of 4 studies (532 participants) assessed postintervention attitude toward educational interventions [
Moreover, 1 study reported improved attitudes following digital education compared with usual learning (SMD=0.45; 95% CI 0.18-0.72) [
A total of 3 studies (415 participants) assessed postintervention satisfaction with the educational interventions using nonvalidated MCQs [
A total of 4 studies (650 participants) assessed postintervention practice and behavior change using nonvalidated surveys [
A total of 6 studies (1988 participants) assessed postintervention self-efficacy using nonvalidated questionnaires [
There was 1 study which assessed postintervention patient outcomes such as smoking index, general health score, quality of life score, and the cost associated with the blended education intervention (
Our review included 11 studies that investigated the effectiveness of digital education for training health professionals to deliver smoking cessation therapy. No difference was found between digital education and traditional or usual learning. There was some suggestion that blended education results in greater improvements in satisfaction, skill, and knowledge compared with digital education alone. There was also some evidence for improved attitude following digital education compared with usual learning. Only 1 study reported patient outcomes and the setup cost of digital education. There were insufficient data to investigate what components of the digital education interventions were associated with the greatest improvements in learning outcomes. Studies were poorly reported, heterogeneous, assessed a broad range of different outcomes, and compared different types of interventions on the range of pre and postregistration health professionals. The findings of this review should therefore be interpreted with some caution.
As far as we are aware, this is the first review to address the topic of digital education to train health professionals to deliver smoking cessation therapy. We followed best practice methods for systematic reviews, which attempted to minimize risk of bias and errors in the review process. We conducted a comprehensive sensitive search across a broad range of databases and included additional steps to identify unpublished studies such as searching trials registers, meeting abstracts and PhD theses, screened references of included studies, and contacted authors of abstracts for further information. It was not possible to formally assess the risk of publication bias because of the small number of heterogeneous studies included in our review, but given our extensive search, we consider it unlikely that relevant studies have been missed. Moreover, 2 independent reviewers were involved in all stages of the review process, standardized data extraction forms were used, and we used an accepted tool to assess the risk of bias in the included studies. This identified potential limitations in the included studies, particularly in reporting, which meant that many of the risk of bias domains were judged as unclear for the majority of studies. The included studies evaluated a broad range of interventions and outcomes; therefore, it was not appropriate to calculate summary effect estimates. The included digital education interventions mostly comprised asynchronous Web-based programs aimed at postregistration health professionals. For busy health professionals, digital education is a convenient avenue for fulfilling continuing medical education requirements and promoting knowledge and skill in particular areas in which they may not have previously had training in. Furthermore, 2 studies included an interactive or feedback component in the digital education intervention [
There were a number of limitations with the included studies. Reporting of the digital education interventions, especially the description of the intervention, aims and outcomes, pedagogical approach, and use of validated outcome assessment instruments was inconsistent across the studies. There was also a lack of baseline assessment in some studies, meaning that only postintervention data could be used in the analysis, potentially biasing results. Several studies did not report numerical data; therefore, they could not be formally included in our synthesis. We have included results available from these studies, but had additional data been available in the included studies, these would have allowed us to conduct a more informative analysis.
When considering the implications of the findings of our review for practice, it is important to consider the implications of the differences in the effectiveness of different types of education. For forms of digital education that have additional benefits compared with standard education (eg, ability to target larger numbers of people, self-paced learning), showing that these are as effective as standard education is likely to be sufficient to recommend the use of these types of education. In contrast, for types of digital education where technology is supplementing standard face-to-face learning (eg, blended education or use of “clickers”), outcomes would need to be better with digital education than with standard education. For example, the study evaluating the use of clickers to give instructions during classroom lectures reported a positive impact of technology on learners’ attitudes, which consequently improved learning outcomes [
There is a need for further robust studies in a range of settings to determine the true potential of digital education to train health professionals to deliver smoking cessation therapy. More studies evaluating patient outcomes such as postintervention smoking cessation rates, smoking status, and abstinence are needed to assess the effectiveness of digital education for health professionals. Before recommending implementation of digital education programs, information on cost effectiveness, sustainability, as well as the direct and indirect costs such as time to develop as well as implement a smoking cessation module is needed. This will help policy makers to make practical recommendations and allocate resources appropriately. More research is needed to understand the feasibility of integrating digital smoking cessation training methods into the curriculum and continued medical education; research is also needed to understand the short-term and long-term effects in different geographical, socioeconomic, and cultural settings. In addition, sustainability, cost savings, and accreditation of digital smoking cessation therapies in health professional training need to be further researched.
Many digital education interventions in the included studies were based on smoking cessation guidelines. However, validation of the course content and use of underpinning learning theories to develop the pedagogy were lacking in most of the studies. Largely, the studies focused on integrating the new technology into the existing curriculum as opposed to using learning theories to design the digital education intervention for successful delivery of education. An increasing body of evidence reveals that theory-based interventions have greater impact than those that are not based on theory. Appropriate use of learning theory and pedagogy framework along with sound methodology can enable more robust studies to be conducted on digital education with research questions adequately addressed through theoretically informed research design, data collection, and analysis [
Digital education can contribute significantly to the World Health Organization (WHO) mission to transform and scale up health professionals’ education by filling the medical education divide between low- and high-income countries [
We did not identify any studies on advanced educational technologies for digital education such as mobile learning, virtual patients, virtual reality environments, or serious gaming, which have the potential to transform education for health professionals. The lack of smoking cessation studies evaluating these educational technologies makes it difficult to make recommendations for integrating such pivotal digital technologies into health professional education. In the future, it is important to have a more detailed reporting of different components of digital education interventions to allow for a more thorough analysis of the most active and effective components.
Digital education appears to be at least as effective as usual or traditional learning in improving health professionals’ knowledge and skill for delivering smoking cessation therapy. This suggests that digital education is an appropriate method to deliver training for health professionals on how to deliver smoking cessation therapy. However, limitations in the evidence base mean that these conclusions should be interpreted with some caution. There was insufficient evidence to determine what components of digital education are associated with the greatest improvements in outcomes, although there was some evidence that blended education may be more effective than either digital education or usual learning alone.
MEDLINE (Ovid) search strategy.
Results on included studies.
cluster randomized trial
multiple-choice question
relative risk
standardized mean difference
World Health Organization
This review was conducted in collaboration with the Health Workforce Department at WHO. We would also like to thank Mr Carl Gornitzki, Ms GunBrit Knutssön, and Mr Klas Moberg from the university library, Karolinska Institutet, Sweden, for developing the search strategy, and the peer reviewers for their comments. We gratefully acknowledge funding from the Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Singapore, eLearning for health professional’s education grant.
LTC conceived the idea for the review. MS and PW wrote the review. RB, SB, and BMK provided comments on the review.
None declared.