Background: There is little consensus regarding effective digital health interventions for diverse populations, which is due in part to the difficulty of quantifying the impact of various media and content and the lack of consensus on evaluating dosage and outcomes. In particular, digital smoking behavior change intervention is an area where consistency of measurement has been a challenge because of emerging products and rapid policy changes. This study reviewed the contents and outcomes of digital smoking interventions and the consistency of reporting to inform future research.
Objective: This study aims to systematically review digital smoking behavior change interventions and evaluate the consistency in measuring and reporting intervention contents, channels, and dose and response outcomes.
Methods: PubMed, Embase, Scopus, PsycINFO, and PAIS databases were used to search the literature between January and May 2021. General and journal-based searches were combined. All records were imported into Covidence systematic review software (Veritas Health Innovation) and duplicates were removed. Titles and abstracts were screened by 4 trained reviewers to identify eligible full-text literature. The data synthesis scheme was designed based on the concept that exposure to digital interventions can be divided into intended doses that were planned by the intervention and enacted doses that were completed by participants. The intended dose comprised the frequency and length of the interventions, and the enacted dose was assessed as the engagement. Response measures were assessed for behaviors, intentions, and psychosocial outcomes. Measurements of the dose-response relationship were reviewed for all studies.
Results: A total of 2916 articles were identified through a database search. Of these 2916 articles, the title and abstract review yielded 324 (11.11%) articles for possible eligibility, and 19 (0.65%) articles on digital smoking behavior change interventions were ultimately included for data extraction and synthesis. The analysis revealed a lack of prevention studies (0/19, 0%) and dose-response studies (3/19, 16%). Of the 19 studies, 6 (32%) reported multiple behavioral measures, and 5 (23%) reported multiple psychosocial measures as outcomes. For dosage measures, 37% (7/19) of studies used frequency of exposure, and 21% (4/19) of studies mentioned the length of exposure. The assessment of clarity of reporting revealed that the duration of intervention and data collection tended to be reported vaguely in the literature.
Conclusions: This review revealed a lack of studies assessing the effects of digital media interventions on smoking outcomes. Data synthesis showed that measurement and reporting were inconsistent across studies, illustrating current challenges in this field. Although most studies focused on reporting outcomes, the measurement of exposure, including intended and enacted doses, was unclear in a large proportion of studies. Clear and consistent reporting of both outcomes and exposures is needed to develop further evidence in intervention research on digital smoking behavior change.
There are few published data on exposure to and evaluations of digital behavior change interventions, and to date, there is little consensus regarding effective interventions for diverse populations. This is because of the difficulty of quantifying digital health interventions that use various media and content and the lack of consensus on how to evaluate and report dose and outcomes. However, digital health is developing rapidly, and understanding the latest evidence is critical for research in this field.
Digital media is central to and an integral part of modern life; however, the study of its effects on health behavior is just beginning . As reported by the World Bank, 45% of the world’s population or 3.5 billion people use social media. Worldwide, the average user spends approximately 3 hours of their day on social media [ ]. Given the widespread exposure to digital technologies such as social media, it is increasingly important to understand how digital media affects individual health decision-making and behavior, as well as social networks and communities. These facts make it critically important to understand how digital media influences behavior. Social and behavioral scientists who study digital media must learn how to design and evaluate effective behavior change interventions, the evidence-based approaches that are effective, and how digital media affects targeted outcomes.
Efforts to understand the practice of digital media interventions have historically been made by public health scientists. Hu  reviewed 348 journal articles and structured the subject, health topics, technologies, and methods used for digital interventions between 2008 and 2012. Abad et al [ ] conducted a scoping review on digital public health surveillance and revealed that only 0.8% of the related studies between 2005 and 2020 deployed a digital health surveillance system that can be used for monitoring and targeted interventions, despite its impact on the study methodology and public health actions. A recent systematic review by Seiler et al [ ] found relatively few rigorous studies on the effectiveness of digital media–related behavior change campaigns and interventions. This review also found that the reporting of design, measures, data collection, and other methods needs to be improved and systematized. Recommendations for improvements included clarification of what is meant by dose and dose-response; how and with what intensity interventions are delivered; and measurement of outcomes, including attitudes, beliefs, social norms, and health behavior. This review also reiterated a previous finding that evidence for behavior change using digital interventions stems primarily from studies conducted in high-income countries [ - ].
Although digital media has been used for a variety of public health programs, one of the most rapidly changing areas is digital smoking behavior change interventions. Owing to the emerging products, devices, and policy changes, it has been more difficult to systematically quantify and evaluate the effectiveness of interventions in this area. Despite the importance of using consistent measures and evaluation methods to understand the impacts of interventions, little effort has been made to understand the common measures and methods used to evaluate the effectiveness of these interventions. There is a need for an assessment of the measures used in this area to inform future research to accurately evaluate the outcomes across a variety of products and devices, as well as rapid policy and market change. To the authors’ knowledge, no previous systematic review has focused on the detailed measurements used in digital smoking interventions.
Although tobacco use has declined overall , it remains at unacceptable levels, exacting personal and social costs, particularly among young adults [ , ]. Tobacco is the leading preventable cause of death in the United States [ ]. In the United States, 18.2% of young adults aged 18 to 24 years reported current use of tobacco products, and 10.4% reported being current cigarette smokers [ ]. Although the age at which smokers initiate cigarette use has been increasing over time, almost all cigarette use initiation occurs before the age of 26 years [ ], making young adults a critical target for prevention efforts. Given the widespread use of digital media among the young adult population [ ], digital interventions may be promising.
To address these gaps in digital smoking behavior change interventions, this review aimed to (1) systematically review and codify the measures used for digital health interventions in tobacco and nicotine use research, (2) evaluate the quality of reporting of dose and response, and (3) identify areas for improvement in the field.
This review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. The protocol was registered in PROSPERO (International Prospective Register of Systematic Reviews; ID CRD42021285655).
The PubMed, Embase, Scopus, PsycINFO, and PAIS databases were used to search the literature. The search terms included “digital intervention,” “health promotion,” “health education,” “health communication,” “digital technology,” “social media,” “social marketing,” “health,” “measures,” “methods,” “frequency,” “impression,” and “reach.” General and journal-based searches were combined to find literature that was directly related to the scope of the study. Specific journals that were searched included The Lancet Digital Health, Journal of Medical Internet Research, Digital Health, Social Networks, npj Digital Medicine, Digital Medicine, Digital Biomarkers, Frontiers in Digital Health, Communication Methods and Measures, Health Technology Assessment, BMC Medical Research Methodology, Computers in Human Behavior, Computers in Biology and Medicine, Journal of Health Communication, Journal of Communication in Healthcare, Health Communication, Health Communication Science Digest, Health Education and Behavior, Digital Medicine, International Journal of Digital Healthcare, Journal of Health and Social Behavior, American Journal of Health Behavior, Journal of Behavioral Health, The Journal of Behavioral Health Services and Research, Health Behavior Research, American Journal of Health Promotion, Health Promotion Practice, Journal of Prevention and Health Promotion, Health Promotion International, International Journal of Health Promotion and Education, and Journal of Health Promotion and Behavior. The literature was searched between January and May 2021, followed by a series of monthly searches to identify additional studies. Search strategies and terms were developed in collaboration with librarians at George Washington University.
All records searched through the database were imported into Covidence systematic review software (Veritas Health Innovation) and duplicates were removed.
Titles and abstracts were screened by 4 trained reviewers to identify eligible full-text literature. Each study was screened by 2 reviewers, and disagreements were resolved through discussion between the reviewers. The inclusion criteria were (1) publication after 2000, (2) full text available in English, (3) peer-reviewed original journal articles, (4) health-related topics, (5) at least one behavior change intervention defined, (6) use of the internet or mobile-based platform for mass or targeted communication, (7) use of digital devices, and (8) measurement of original data related to behavior or psychosocial measures.
Behavior change interventions were defined as planned programs that had stated objectives related to behavior change, target populations, and targeted messages in text, audio, video, graphics, or other distributed forms in a one-to-many format. Studies that assessed at least one behavioral or psychosocial measure defined in the social behavioral theories were included. Studies that only measured engagement were excluded. Digital media is defined as an internet-based platform for mass and targeted communications, including social media, apps, websites, software, blogging, and one-to-one chat platforms used for mass and targeted communications (eg, WhatsApp). Video games, emails, radio, and television were excluded from digital media. Studies that used digital media as a channel for one-to-one communication such as conversations between health care providers and patients were also excluded.
After literature was screened using the abovementioned criteria, articles related to smoking were extracted for this review. The terms used for this process included “tobacco,” “smoking,” “cigarette,” “vaping,” “vape,” “e-cig,” “ENDS,” “nicotine,” “hookah,” “JUUL,” “cigar,” “e-liquid,” “flavor,” “smokeless,” “smoker,” and “vaper.” The literature search was repeated replacing the term “health” with “smoking” and “vaping” on the database, and we checked for the coverage of the articles on the abovementioned topics.
Data Extraction and Synthesis
The data extraction and synthesis scheme was designed to identify the format of the digital media intervention, the measurement of each component that assessed dose and response in the intervention, and the study design used in the research. Dose measurement items were developed based on the concept that exposure to digital health interventions can be divided into 2 parts . The first part is the intended dose, which refers to planned exposure by the intervention side. The other part is an enacted dose that corresponds to a portion of the intended dose that is actually completed by the participant. The dose measurement items were designed so that doses can be expressed as the frequency of the intervention multiplied by the length of intervention component and amount of engagement, which offers supplemental information about active involvement by participants. In this review, the intended dose comprises the frequency and length of the intervention, and the enacted dose comprises engagement. Engagement was defined as the interaction between the intervention content and the participants, such as views, clicks, likes, comments, and shares. Response equals the outcomes of the intervention, including behavior, intention, and other psychosocial factors that were previously confirmed to be connected in social behavioral change theories. A dose-response relationship was defined as the association between different levels of doses (exposure) and responses (outcomes), and its application was assessed in this review.
The codebook for data extraction and synthesis was developed in a Microsoft Excel format and piloted using 20 randomly selected articles on digital health behavior change interventions. The extracted data included basic information about the study, types of digital media and devices used, modes of intervention, measures used for intervention exposure, outcomes, engagement, study designs, model applications, cost and funding information, and source of bias. Data were converted into standardized forms where necessary and checked for clarity of reporting. The codebook had 3 types of input formats, including categorical options with data validation; hence, coders could only type prespecified responses, dichotomous options with data validation that indicated yes or no, and free answers where coders could leave notes. Coders were the same as the reviewers in the screening phase, who had 4 weeks of training using this codebook. The coding for this review started after the interrater reliability met 80% agreement in the training. Disagreements were resolved by discussion. The specific items extracted and synthesized are summarized in the Results section.
A total of 2916 articles were identified through a database search, of which 2797 (95.92%) records were screened for titles and abstracts after duplicates were removed. Title and abstract review of the 2797 articles yielded 324 (11.58%) articles for possible eligibility, and 253 (9.05%) articles were screened for the subject matter. Ultimately, of the 253 articles, 19 (7.5%) articles on digital smoking behavior change interventions were included for data extraction and synthesis. The flow of literature screening is presented in.
A summary of digital smoking behavior change interventions is presented in. Of the 19 studies included, 13 (68%) focused on smoking cessation and some also included smoking reduction as a secondary outcome that led to smoking cessation. Several articles focused on promoting a social movement for rejecting tobacco and reducing the influence of peer smoking, and others had more general topics such as promoting healthy lifestyles, tobacco-free lifestyles, and antitobacco norms.
Of the 19 studies, 10 (53%) studies used multiple digital media platforms. Websites, apps, and social media were the most frequently used channels. Among the studies that used single digital media, Facebook (3/19, 16%), apps (3/19, 16%), websites (1/19, 5%), YouTube (1/19, 5%), and software (1/19, 5%) were used. Multiple devices, including laptops, tablets, and smartphones, were used in 53% (10/19) of the studies. Of the 19 studies, 2 (11%) used smartphones, and 1 (5%) study each used tablets, cell phones, and a special device developed for the intervention, respectively. Approximately 21% (4/19) of studies did not explicitly report the devices used.
Of the 19 studies, 9 (53%) combined multiple modes of communication for the intervention. SMS text messages, images, and videos were typically used. Studies focusing on a single mode used text messages (3/19, 16%), videos (1/19, 5%), or images (1/19, 5%). Approximately 16% (3/19) of the studies were unclear about the modes of communication.
|Source||Topic||Digital media||Device||Mode||Theoretical model||Conceptual model|
|Baskerville et al , Canada||Smoking cessation||Website, app, Facebook, and YouTube||Laptop and smartphone||Video, image, and articles||No||No|
|Davis et al , United States||Smoking cessation||YouTube and television||Desktop or laptop, mobile device, and television||Digital video advertisements and television advertisements||No||No|
|Goldenhersch et al , Argentina||Smoking cessation||App and cardboard headset device||Smartphone||Video, audio, and group chat||No||No|
|Guillory et al , United States||Tobacco-free lifestyle||Digital and social media (not reported specifically), radio, print, out of home, and local events||Laptop and smartphone||Unclear||Yes||No|
|Hair et al , United States||Antitobacco social movement||Social media, website, YouTube, and television||Laptop, tablet, and smartphone||Unclear||No||No|
|Kenfield et al , United States||Smoking as part of multiple themes||Website, Fitbit One, and SMS text messaging||Laptop, smartphone, and FitBit||SMS text message||No||No|
|Kim et al , United States||Smoking cessation||Website (video, mobile, and search advertisements that direct the campaign website) and display||Laptop, tablet, and smartphone||Video and image||No||No|
|Kim et al , United States||Smoking cessation and reduction||Laptop and smartphone||Video-, text- and, image-based materials||No||No|
|An et al , United States||Smoking as part of multiple themes||Websites||Laptop and phone||SMS text message, image, video, and phone call||Yes||No|
|Marler et al , United States||Smoking cessation and reduction||Apps||Other (The Pivot Breath Sensor, a mobile Pivot app)||SMS text message||Yes||No|
|Masaki et al , Japan||Smoking cessation||App, a connected cloud system, a paired mobile exhaled carbon monoxide checker device, and a web-based PC software||Smartphone and laptop||Digital diary, videos, chatbot, and biomedical recording||No||No|
|Namkoong et al , United States||Antitobacco social movement||Not reported||Videos, text, and pictures||Yes||Yes|
|Onezi et al , Saudi Arabia||Smoking cessation (smoking relapse prevention)||Twitter and WhatsApp||Not reported||Other (social media–based support groups)||No||No|
|Romer et al , United States||Reduction of influence of peer smoking||YouTube||Not reported||Video||No||No|
|Thrul and Ramo , United States||Smoking cessation||Laptop, tablet, and smartphone||Unclear||Yes||No|
|Tsoh et al , United States||Smoking cessation||Tablet||Assessment, video messages, 1-page summary printout, or email||No||No|
|Webb et al , United Kingdom||Smoking cessation||App||Smartphone||Video, audio, quizzes, and quit coach through digital chat||No||No|
|Bary-Weisberg et al , Israel||Smoking cessation||Website and SMS text messaging||Cell phone (for SMS text messaging)||SMS text message||No||No|
|Burford et al , Australia||Smoking cessation||Other (The APRIL Face Aging software)||Not reported||Image||Yes||No|
Approximately 68% (13/19) of the articles stated that specific theoretical models applied to interventions, whereas 32% (6/19) did not report any models. These models were defined as previously published theoretical models related to social, cognitive, and behavioral factors. Specific models mentioned in the literature include the social cognitive theory , transtheoretical models [ ], theory of reasoned action, d theory of planned behavior, self-determination theory [ , ], and social branding framework. This study also reviewed the application of conceptual models defined as frameworks designed for specific interventions in the literature. Approximately 95% (18/19) of the studies did not mention any conceptual model specific to the interventions.
Measures of Dose and Response
The measures used to assess the dose (exposures), response (outcomes), and form of measurement are summarized in. For response measures, 32% (6/19) of studies reported multiple behavioral outcomes. Measures included smoking status, smoking reduction, abstinence, quit attempts, successful quitting, information search related to smoking cessation, campaign-related topics, and the use of cessation aids. Of the 19 studies, 5 (23%) used multiple psychosocial measures, and 4 (21%) reported a single psychosocial measure. Approximately 37% (7/19) of studies did not report any psychosocial measures. Intention (4/19, 21%), self-efficacy (4/19, 21%), awareness (3/19, 16%), norms (2/19, 11%), and stage of change (2/19, 11%) were mainly used.
For dose measurements, the frequency of exposure, length of exposure, and engagement were assessed. This was designed under the assumption that the amount of dose can be expressed as the sum of the intended dose and the enacted dose; the intended dose equals the frequency of exposure multiplied by the length of exposure planned by the intervention side, and the enacted dose equals the amount of engagement that was actively received by the targeted. Approximately 37% (7/19) of studies explicitly reported the frequency of exposure, whereas 63% (12/19) of studies were unclear on that point. Of the 19 studies, 3 (16%) studies reported the frequency per week, 3 (16%) other studies reported the frequency per day, and 2 (11%) studies reported that the frequency varied for each week. The length of exposure was explicitly reported in 21% (4/19) of studies, whereas 79% (15/19) of studies were unclear about this. All 4 studies that mentioned the length of exposure used videos for the intervention. Engagement was measured in 58% (11/19) of studies. Of the 19 studies, 8 (42%) reported multiple measures, and 3 (16%) reported a single measure. Visits, clicks, log-ins, and views were measured most frequently across different modes of the intervention. Social media interventions reported subscriptions, likes, comments, and postings of content. Finally, 42% (8/19) of studies did not report on engagement.
The dose-response relationship was assessed in 16% (3/19) of studies using different levels of exposure. One of the studies compared the effects of intervention between a standard-dose group and a higher-dose group. Another study examined the interaction between the time of exposure and the treatment arm. One of the studies examined the relationship between participants’ levels of active engagement and the targeted behavior. Outcome assessment was self-reported in 68% (13/19) of studies and a combination of self-report and objective measurements in 21% (4/19) of studies. One of the studies applied only objective measurements using tracking software, and another study used an aggregated self-report measure that assessed population-level awareness.
|Source||Response (outcome)||Dose (exposure)||Dose-response||Outcome report|
|Behavioral||Psychosocial||Frequency of exposure||Length of exposure||Engagement|
|Baskerville et al , Canada||Smoking cessation and use of cessation aid||Intention to quit||No||No||Visit, installation of the app, and posting content||No||Self-reported|
|Davis et al , United States||Not measured||Advertisement awareness||No (gross rating points used); depends on the size of the market||Yes (30-second advertisements)||Not measured||Yes||Self-reported|
|Goldenhersch et al , Argentina||Abstinence||Readiness to quit||Yes (every day)||Yes (10-minute videos)||Not measured||No||Self-reported|
|Guillory et al , United States||Not measured||Awareness and receptivity||No||No||Not measured||Yes (time×treatment interaction)||Self-reported|
|Hair et al , United States||Current cigarette use||Advertisement awareness and intentions||No||No||View||No||Other; aggregated self-reported advertisement recall across people grouped by time (weeks) to form a measure of advertisement awareness|
|Kenfield et al , United States||Smoking||Not measured||Yes (4-5 SMS text messages each week)||No||Activity data from Fitbit, response to SMS text messages, website log-in, and page view||No||Self-reported|
|Kim et al , United States||Campaign-related topics search||Not measured||No||No||Visit, impressions, and clicks||No||Measured objectively|
|Kim et al , United States||Smoking reduction||Antismoking attitudes, readiness to quit, motivation to quit, self-efficacy beliefs, and perceived social support||Yes (different each week)||No||Likes and comments||Yes||Self-reported|
|An et al , United States||30-day abstinence from cigarette smoking||Not measured||Yes (weekly)||No||Not measured||No||Self-reported|
|Marler et al , United States [ ]||Quit attempts, cigarettes per day reduction, and abstinence||Stage of change, desire to quit, readiness to quit, confidence to quit, difficulty to quit, and goals||Yes (>4 times use a day); up to twice weekly SMS text messages||No||Not measured||No||Both|
|Masaki et al , Japan||Smoking cessation||Not measured||No||No||Not measured||No||Both|
|Namkoong et al , United States||Smoking-related information seeking||Attitude, descriptive norms, subjective norms, behavioral control, and behavioral intention||Yes (every day)||No||Likes and comments||No||Self-reported|
|Onezi et al , Saudi Arabia||Smoking cessation and smoking frequency||Not measured||No||No||Subscription to a social media support group||No||Self-reported|
|Romer et al , United States||Not measured||Smoking norms, mortality beliefs, and smoking attitudes||No||Yes (approximately 4-5 seconds and display of messages)||Not measured||No||Self-reported|
|Thrul and Ramo , United States||Purposeful 24-hour smoking quit attempt||Not measured||No||No||Not measured||No||Self-reported|
|Tsoh et al , United States||Smoking abstinence, 24-hour quit attempts, and quit methods||Not measured||No||Yes (videos ranged from 8 to 65 seconds, averaging 29 seconds in length); patients watched 14 to 22 video segments depending on their responses||Session completion and patient-provider discussion||No||Both|
|Webb et al , United Kingdom||Smoking status, self-reported 7-day point prevalence abstinence at 4 weeks after the quit date, 14-day point prevalence abstinence, and any additional quit attempts after the quit date||Attitudes and perceptions of smoking; self-reported changes in confidence levels, knowledge, attitudes, and perceptions related to smoking cessation; and changes in Smoking Abstinence Self-efficacy questionnaire||No||No||App opens, stage progression, number of messages sent, check-ins, and diary entries||No||Self-reported|
|Bary-Weisberg et al , Israel||Smoking status||Self-efficacy||Yes (different each week)||No||Keywords sent on text||No||Self-reported|
|Burford et al , Australia||Successful quitting and quit attempts||Progression along the transtheoretical stages of change model and self-perceptions and attitudes toward smoking behavior||No||No||Not measured||No||Both|
Quality of Reporting
The clarity of reporting was assessed for media role, dose and response measurement, and funding sources. The findings on clarity of reporting, study design, and bias are summarized in.
The role of media was clearly reported in all the studies (19/19, 100%). Behavior was clearly measured in all 15 studies that assessed behavioral outcomes. Among the 12 studies that assessed psychosocial outcomes, 11 (92%) reported them clearly, whereas 1 (8%) was unclear about its measurement.
|Source||Clarity of media role||Response (outcomes)||Dose (exposure)||Study design||Bias||Funding source|
|Behavior report||Psychosocial report||Intervention duration||Data collection duration||Engagement report|
|Baskerville et al , Canada||Yes||Yes||Yes||Unclear||2 to <3 months||Yes||Quasi-experimental study||Generalizability and self-report||Yes|
|Davis et al , United States||Yes||N/Aa||Yes||Unclear||Unclear||N/A||Nonexperimental study||Randomization and self-report||Yes|
|Goldenhersch et al , Argentina||Yes||Yes||Yes||2 weeks to <1 month||2 to <3 months||Yes||Experimental study||Short period of data collection||Yes|
|Guillory et al , United States||Yes||N/A||Yes||Unclear||>1 year||N/A||Quasi-experimental study||Aided awareness report (recall bias)||Yes|
|Hair et al , United States||Yes||Yes||Yes||Unclear||>1 year||No||Nonexperimental study||Representativeness||Yes|
|Kenfield et al , United States||Yes||Yes||N/A||2 to <3 months||2 to <3 months||Yes||Experimental study||Representativeness||Yes|
|Kim et al , United States||Yes||Yes||N/A||2 to <3 months||2 to <3 months||Yes||Nonexperimental study||Generalizability||Yes|
|Kim et al , United States||Yes||Yes||Yes||2 weeks to <1 month||1 to <2 months||Yes||Nonexperimental study||Self-report, small sample size, and representativeness||Yes|
|An et al , United States||Yes||Yes||N/A||1 to <2 months||2 to <3 months||N/A||Experimental study||Self-report (recall bias)||Yes|
|Marler et al , United States||Yes||Yes||Yes||2 to <3 months||2 to <3 months||N/A||Nonexperimental study||Generalizability||Yes|
|Masaki et al , Japan||Yes||Yes||N/A||4 to <6 months||6 months to <1 year||N/A||Experimental study||Representativeness||Yes|
|Namkoong et al , United States||Yes||N/A||Yes||2 weeks to <1 month||2 weeks to <1 month||No||Quasi-experimental study||Representativeness||No|
|Onezi et al , Saudi Arabia||Yes||Yes||N/A||Not reported||Not reported||Yes||Nonexperimental study||Generalizability and cross-sectional||Yes|
|Romer et al , United States||Yes||N/A||Yes||Not reported||Not reported||N/A||Quasi-experimental study||Generalizability and control setting||Yes|
|Thrul and Ramo , United States||Yes||Yes||N/A||1 to <2 months||6 months to <1 year||N/A||Nonexperimental study||Self-report (recall bias), representativeness, and low test power||Yes|
|Tsoh et al , United States||Yes||Yes||N/A||Other||2 to <3 months||Yes||Nonexperimental study||No control and self-report||Yes|
|Webb et al , United Kingdom||Yes||Yes||Yes||2 weeks to <1 month||6 months to <1 year||Yes||Experimental study||Generalizability and biased sample||Yes|
|Bary-Weisberg et al , Israel||Yes||Yes||Yes||4 to <6 months||Unclear||Yes||Nonexperimental study||Representativeness||Yes|
|Burford et al , Australia||Yes||Yes||No||Not reported||4 to <6 months||N/A||Experimental study||No blinding||No|
aN/A: not applicable.
The duration of intervention was unclear in 21% (4/19) of studies and was not reported in 16% (3/19) of studies. Of the 19 studies, 12 (63%) reported this explicitly. The intervention duration ranged from 10 minutes to 6 months. One of the studies (1/12, 8%) reported it in the range as it varied among participants because of the customization function of the digital intervention. Of the 19 studies, the intervention duration was 2 weeks to a month in 4 (33%) studies, 1 to 2 months in 2 (17%) studies, 2 to 3 months in 3 (25%) studies, and 4 to 6 months in 2 (17%) studies. The duration of data collection was unclear in 11% (2/19) of studies and was not reported in 11% (2/19) of studies. It ranged from 2 to 3 months in 47% (7/15) of studies, 6 months to a year in 20% (3/15) of studies, and >1 year in 13% (2/15) of studies. Approximately 7% (1/15) of studies each collected data for 2 weeks to a month, 1 to 2 months, and 4 to 6 months, respectively. The reporting of engagement was clear in 82% (9/11) of the studies that assessed engagement.
Of the 19 studies, 6 (32%) applied experimental study designs, 4 (21%) used quasi-experimental designs, and 9 (47%) used nonexperimental study designs. Quasi-experimental designs were defined as studies with a control group that did not involve random assignment. Of the 6 experimental studies, 3 (50%) reported representativeness and a biased sample as a potential source of bias. Quasi-experimental studies reported that self-reporting and aided recall were threats to bias (2/4, 50%). Lack of randomization, self-reporting, small sample size, and lower statistical power were frequently reported, in addition to generalizability and representativeness among nonexperimental studies. Of the 19 studies, 17 (89%) reported a source of funding, and 2 (11%) did not report this information.
This study identified the literature on digital behavior change interventions related to smoking and addressed the current practice of measuring and reporting intervention contents, channels, and dose and response outcomes. Data synthesis showed that both measurement and reporting were inconsistent across studies, illustrating the current challenges in this field of research.
This review revealed a lack of preventive studies on tobacco and nicotine use. Among 324 digital behavior change intervention papers, only 19 (5.9%) papers were relevant to smoking, and none centered on preventing its initiation. Although a literature search detected numerous prevention research papers, this review only included papers that assessed behavioral and psychosocial outcomes and did not include studies that focused on perceptions and engagement. This lack of research may reflect the fact that prevention studies for digital interventions remain at an early stage of identifying the basic conditions to make interventions engaging and effective rather than measuring the effectiveness of such interventions in achieving behavior change. This may also be because of a lack of funding in prevention research, as well as challenges in long-term follow-up to detect differences in smoking initiation rates.
Another gap in the literature on digital health identified in this review was the dose-response relationship. Although the effectiveness of interventions depends on the amount of exposure, only 16% of the literature has assessed the outcomes across different levels of doses.
The biggest issue was the inconsistency and vagueness of the reporting. Most studies paid attention to reporting outcomes and were relatively clear on their measurements in the methods sections. However, reporting of the amount of intervention offered to participants and the actual engagement was often unclear and not explicitly mentioned in a large proportion of studies. Insufficient details on the amount of exposure make it more difficult to compare outcomes across studies and conduct meta-analyses, which hinders the provision of evidence on effective digital interventions.
These findings were consistent with previous research on digital health interventions. Hu et al  found that only 10.6% of papers on digital health focused on health promotion and interventions, and only 2.6% centered on substance use, including tobacco use. Abad et al [ ] pointed out that only 6.8% of the literature on digital health focused on smoking. Another review by Seiler et al [ ] demonstrated that there have been relatively few rigorous studies on the effectiveness of digital behavior change interventions, and the reporting of design, measures, data collection, and other methods need to be improved [ ].
The amount of exposure to an intervention can be divided into the intended dose planned by the intervention side and the enacted dose that was completed by the participant side. Dose can be expressed as the frequency multiplied by the length of the intended intervention dosage, and the amount of engagement can offer information about the enacted dosage. Clarifying these elements in the literature will lead to the promotion of comparable reporting and advancement of the evidence development of digital health interventions.
The types of behaviors used to report the outcomes were mixed. It can be divided into (1) targeted behavior (ie, smoking cessation), (2) surrogate behavior (ie, smoking reduction), and (3) behavior related to the improvement of the likelihood of conducting targeted behaviors (ie, use of cessation aid and smoking cessation information search). Although the measurement of these outcomes was predominantly self-reported, some studies combined objective measurements. The methods of objective measurement included (1) biochemical devices (eg, exhaled carbon monoxide tracker), (2) automatic digital tracking (eg, for information searches), (3) electronic health records, and (4) group-level psychological measures (eg, group-level awareness of campaigns). Leveraging these emerging methods will enhance the validity and reliability of measurements and advance evidence-based digital health interventions.
To the best of our knowledge, this is the first study to review the dose and response measurements of digital smoking interventions. Dose measures were divided into the intended dose from the intervention side and the enacted dose that was actually completed by the participant side. The intended dose was separately assessed for frequency and length. Response measures were organized into behavioral and psychosocial outcomes, and measurement methods were assessed to determine their validity. The proportion of dose-response relationship studies was identified to determine the stage of current research in the field. The literature search combined general and journal-specific searches and yielded 2916 studies that showed high coverage.
Research gaps exist in assessing digital behavior change interventions with clear dose and response measurements and the dose-response relationship between the levels of intervention exposure and outcomes. More prospective studies are needed to examine the relationship between higher and lower dosages of interventions on smoking outcomes. For example, a well-designed dose-response relationship study on vaping outcomes among a specific population will identify the effective amount of digital intervention to prevent the initiation and decrease the amount of e-cigarette use. This will provide evidence for identifying the effective amount of intervention and offer grounds for conducting well-designed meta-analyses that synthesize evidence for digital behavior change interventions. This paper contributes to building a base for these studies.
This review had several limitations. First, the number of studies included in this analysis was small. This was mainly because of the lack of digital behavior change intervention studies related to smoking and the measurement of behavioral and psychosocial outcomes in the current phase in the area. Second, the risk of bias assessment was omitted from the review. This was because of the nature of the mixed study designs in the included studies as there is little risk of bias assessment tools intended to apply to a variety of study designs. Instead, the quality of the papers was assessed by reviewing and evaluating the study design, theoretical model application, conceptual model application, cost, funding source, and bias source. In addition, the clarity of the media role and every measure of dose and response were assessed as part of the review for each study. When digital smoking behavior change interventions become more common and more literature can be included in the future, these limitations can be minimized.
There are challenges in every emerging area of scientific research. Digital smoking interventions are a new and growing area of research involving continuous and rapid advancements in technologies for health intervention delivery, implementation, and dissemination. This characteristic implies that it is essential to use consistent, standardized methods to evaluate the outcomes to accurately understand the efficacy and effectiveness of the interventions in rapid change, particularly in this area. For future improvement in this realm, this study assessed the consistency of measures used to quantify the dose and response in digital smoking interventions. These results suggest that both dose and response measures are often not clearly defined and are inconsistent across studies. This can imply that synthesizing the evidence on digital smoking interventions has been a challenge, and past studies could have been based on vague reporting and outcome evaluations. This is particularly an issue as interventions to date could be based on biased findings.
This study details and structures the composition of evaluation measures and assesses their use in digital smoking intervention studies. To the best of our knowledge, this is the first study to review dose and response measurements of digital smoking interventions. Although it has limitations such as the small number of papers included and the limited risk of bias assessment, the study successfully (1) reviewed and codified measures used for digital health interventions in tobacco and nicotine use research, (2) evaluated the quality of reporting of dose and response, and (3) identified areas for improvement in the field. More prospective studies that examine the clarity and consistency of measures among a larger number of studies are needed to develop further grounds for evidence-based digital smoking interventions. There is also a need for a clear and consistent reporting scheme for digital health interventions to accurately evaluate outcomes and conduct well-designed meta-analyses. Provision of clear and consistent reporting of both outcomes and exposures is needed to develop further evidence in this field that leads to protecting the lives and health of the public.
This study was supported by the National Cancer Institute (R01CA253013; principal investigator: WDE).
MI and WDE conceived the study. MI designed the study protocol. WDE and JC contributed to the study design and provided expertise in digital media studies on tobacco control. MI, RG, SM, and AB performed the analyses. MI prepared the manuscript. All authors contributed to the final approval of the manuscript.
Conflicts of Interest
- Seiler J, Libby TE, Jackson E, Lingappa J, Evans W. Social media-based interventions for health behavior change in low- and middle-income countries: systematic review. J Med Internet Res 2022 Apr 14;24(4):e31889 [FREE Full text] [CrossRef] [Medline]
- Sanchez-Paramow C, Legovini A. Using social media to change norms and behaviors at scale. World Bank Blogs. 2021 Jan 12. URL: https://blogs.worldbank.org/voices/using-social-media-change-norms-and-behaviors-scale [accessed 2022-03-28]
- Hu Y. Health communication research in the digital age: a systematic review. J Commun Healthc 2015 Dec 18;8(4):260-288. [CrossRef]
- Shakeri Hossein Abad Z, Kline A, Sultana M, Noaeen M, Nurmambetova E, Lucini F, et al. Digital public health surveillance: a systematic scoping review. NPJ Digit Med 2021 Mar 03;4(1):41 [FREE Full text] [CrossRef] [Medline]
- Elaheebocus SM, Weal M, Morrison L, Yardley L. Peer-based social media features in behavior change interventions: systematic review. J Med Internet Res 2018 Feb 22;20(2):e20 [FREE Full text] [CrossRef] [Medline]
- Hsu MS, Rouf A, Allman-Farinelli M. Effectiveness and behavioral mechanisms of social media interventions for positive nutrition behaviors in adolescents: a systematic review. J Adolesc Health 2018 Nov;63(5):531-545. [CrossRef] [Medline]
- Chau M, Burgermaster M, Mamykina L. The use of social media in nutrition interventions for adolescents and young adults-a systematic review. Int J Med Inform 2018 Dec;120:77-91 [FREE Full text] [CrossRef] [Medline]
- Brigden A, Anderson E, Linney C, Morris R, Parslow R, Serafimova T, et al. Digital behavior change interventions for younger children with chronic health conditions: systematic review. J Med Internet Res 2020 Jul 31;22(7):e16924 [FREE Full text] [CrossRef] [Medline]
- Chan L, O'Hara B, Phongsavan P, Bauman A, Freeman B. Review of evaluation metrics used in digital and traditional tobacco control campaigns. J Med Internet Res 2020 Aug 11;22(8):e17432 [FREE Full text] [CrossRef] [Medline]
- Miech R, Johnston L, Bachman J, O?Malley P, Schulenberg J, Patrick M. Monitoring the future: a continuing study of American youth (12th-Grade Survey), 2020. Inter-university Consortium for Political and Social Research. 2021 Oct 26. URL: https://doi.org/10.3886/ICPSR38156.v1 [accessed 2022-05-15]
- Truth initiative homepage. Truth Initiative. URL: http://truthinitiative.org/ [accessed 2022-05-15]
- Preventing Tobacco Use Among Youth and Young Adults A Report of the Surgeon General. Pittsburgh, PA: U.S. Department of Health and Human Services, Public Health Service, Office of the Surgeon General; 2012.
- Health effects of cigarette smoking. Centers for Disease Control and Prevention. URL: https://www.cdc.gov/tobacco/data_statistics/fact_sheets/health_effects/effects_cig_smoking/index.htm#cancer [accessed 2022-05-15]
- Cornelius ME, Wang TW, Jamal A, Loretan CG, Neff LJ. Tobacco product use among adults - United States, 2019. MMWR Morb Mortal Wkly Rep 2020 Nov 20;69(46):1736-1742 [FREE Full text] [CrossRef] [Medline]
- Auxier B, Anderson M. Social media use in 2021. Pew Research Center. 2021 Apr 7. URL: https://www.pewresearch.org/internet/2021/04/07/social-media-use-in-2021/ [accessed 2022-05-15]
- McVay MA, Bennett GG, Steinberg D, Voils CI. Dose-response research in digital health interventions: concepts, considerations, and challenges. Health Psychol 2019 Dec;38(12):1168-1174 [FREE Full text] [CrossRef] [Medline]
- Baskerville NB, Azagba S, Norman C, McKeown K, Brown KS. Effect of a digital social media campaign on young adult smoking cessation. Nicotine Tob Res 2016 Mar;18(3):351-360. [CrossRef] [Medline]
- Davis KC, Shafer PR, Rodes R, Kim A, Hansen H, Patel D, et al. Does digital video advertising increase population-level reach of multimedia campaigns? Evidence from the 2013 tips from former smokers campaign. J Med Internet Res 2016 Sep 14;18(9):e235 [FREE Full text] [CrossRef] [Medline]
- Goldenhersch E, Thrul J, Ungaretti J, Rosencovich N, Waitman C, Ceberio MR. Virtual reality smartphone-based intervention for smoking cessation: pilot randomized controlled trial on initial clinical efficacy and adherence. J Med Internet Res 2020 Jul 29;22(7):e17571 [FREE Full text] [CrossRef] [Medline]
- Guillory J, Henes A, Farrelly MC, Fiacco L, Alam I, Curry L, et al. Awareness of and receptivity to the fresh empire tobacco public education campaign among hip hop youth. J Adolesc Health 2020 Mar;66(3):301-307. [CrossRef] [Medline]
- Hair EC, Niederdeppe J, Rath JM, Bennett M, Romberg A, Pitzer L, et al. Using aggregate temporal variation in ad awareness to assess the effects of the truth® campaign on youth and young adult smoking behavior. J Health Commun 2020 Mar 03;25(3):223-231. [CrossRef] [Medline]
- Kenfield SA, Van Blarigan EL, Ameli N, Lavaki E, Cedars B, Paciorek AT, et al. Feasibility, acceptability, and behavioral outcomes from a technology-enhanced behavioral change intervention (prostate 8): a pilot randomized controlled trial in men with prostate cancer. Eur Urol 2019 Jun;75(6):950-958. [CrossRef] [Medline]
- Kim A, Hansen H, Duke J, Davis K, Alexander R, Rowland A, et al. Does digital ad exposure influence information-seeking behavior online? Evidence from the 2012 tips from former smokers national tobacco prevention campaign. J Med Internet Res 2016 Mar 16;18(3):e64 [FREE Full text] [CrossRef] [Medline]
- Kim SJ, Marsch LA, Brunette MF, Dallery J. Harnessing Facebook for smoking reduction and cessation interventions: Facebook user engagement and social support predict smoking reduction. J Med Internet Res 2017 May 23;19(5):e168 [FREE Full text] [CrossRef] [Medline]
- An LC, Demers MR, Kirch MA, Considine-Dunn S, Nair V, Dasgupta K, et al. A randomized trial of an avatar-hosted multiple behavior change intervention for young adult smokers. J Natl Cancer Inst Monogr 2013 Dec;2013(47):209-215 [FREE Full text] [CrossRef] [Medline]
- Marler JD, Fujii CA, Wong KS, Galanko JA, Balbierz DJ, Utley DS. Assessment of a personal interactive carbon monoxide breath sensor in people who smoke cigarettes: single-arm cohort study. J Med Internet Res 2020 Oct 02;22(10):e22811 [FREE Full text] [CrossRef] [Medline]
- Masaki K, Tateno H, Nomura A, Muto T, Suzuki S, Satake K, et al. A randomized controlled trial of a smoking cessation smartphone application with a carbon monoxide checker. NPJ Digit Med 2020;3:35 [FREE Full text] [CrossRef] [Medline]
- Namkoong K, Nah S, Record RA, Van Stee SK. Communication, reasoning, and planned behaviors: unveiling the effect of interactive communication in an anti-smoking social media campaign. Health Commun 2017 Jan;32(1):41-50. [CrossRef] [Medline]
- Onezi HA, Khalifa M, El-Metwally A, Househ M. The impact of social media-based support groups on smoking relapse prevention in Saudi Arabia. Comput Methods Programs Biomed 2018 Jun;159:135-143. [CrossRef] [Medline]
- Romer D, Jamieson PE, Jamieson KH, Jones C, Sherr S. Counteracting the influence of peer smoking on YouTube. J Health Commun 2017 Apr;22(4):337-345 [FREE Full text] [CrossRef] [Medline]
- Thrul J, Ramo DE. Cessation strategies young adult smokers use after participating in a Facebook intervention. Subst Use Misuse 2017 Jan 28;52(2):259-264 [FREE Full text] [CrossRef] [Medline]
- Tsoh JY, Quach T, Duong TB, Sa Nan Park E, Wong C, Huang SM, et al. Interactive mobile doctor (iMD) to promote patient-provider discussion on tobacco use among Asian American patients in primary care: a pilot study. J Am Board Fam Med 2018 Nov 09;31(6):869-880 [FREE Full text] [CrossRef] [Medline]
- Webb J, Peerbux S, Smittenaar P, Siddiqui S, Sherwani Y, Ahmed M, et al. Preliminary outcomes of a digital therapeutic intervention for smoking cessation in adult smokers: randomized controlled trial. JMIR Ment Health 2020 Oct 06;7(10):e22833 [FREE Full text] [CrossRef] [Medline]
- Bary-Weisberg D, Meltser M, Oberman M, Pato Benari A, Bar-Zeev Y, Shalev S, et al. Feasibility of a text-messaging smoking cessation program for soldiers in Israel. BMC Public Health 2019 Jun 26;19(1):715 [FREE Full text] [CrossRef] [Medline]
- Burford O, Jiwa M, Carter O, Parsons R, Hendrie D. Internet-based photoaging within Australian pharmacies to promote smoking cessation: randomized controlled trial. J Med Internet Res 2013 Mar 26;15(3):e64 [FREE Full text] [CrossRef] [Medline]
- Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev 1977;84(2):191-215. [CrossRef]
- Prochaska JO, DiClemente CC, Norcross JC. In search of how people change: applications to addictive behaviors. Am Psychol 1992;47(9):1102-1114. [CrossRef]
- Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process 1991 Dec;50(2):179-211. [CrossRef]
- Fishbein M. A theory of reasoned action: some applications and implications. Nebr Symp Motiv 1980;27:65-116. [Medline]
|PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses|
|PROSPERO: International Prospective Register of Systematic Reviews|
Edited by R Kukafka; submitted 04.04.22; peer-reviewed by H Mehdizadeh, N Chekin; comments to author 03.05.22; revised version received 27.06.22; accepted 25.07.22; published 25.08.22Copyright
©Megumi Ichimiya, Raquel Gerard, Sarah Mills, Alexa Brodsky, Jennifer Cantrell, W Douglas Evans. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 25.08.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.