This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Web-based programs for smoking prevention are being increasingly used with some success among adolescents. However, little is known about the mechanisms that link the experience of such programs to intended nicotine or tobacco control outcomes.
Based on the experiential learning theory and extended elaboration likelihood model, this study aimed to evaluate the impact of a Web-based intervention, A Smoking Prevention Interactive Experience (ASPIRE), on adolescents’ intention to smoke, while considering the experience of interactivity and entertainment as predictors of reduced intention to smoke, under a transitional user experience model.
A total of 101 adolescents were recruited from after-school programs, provided consent, screened, and randomized in a single-blinded format to 1 of 2 conditions: the full ASPIRE program as the experimental condition (n=50) or an online , text-based version of ASPIRE as the control condition (n=51). Data were collected at baseline and immediate follow-up. Repeated-measures mixed-effects models and path analyses were conducted.
A total of 82 participants completed the study and were included in the analysis. Participants in the experimental condition were more likely to show a decrease in their intention to smoke than those in the control condition (beta=−0.18,
Adolescents’ experience of interactivity and entertainment contributed to the expected outcome of lower intention to smoke. Also, emphasis needs to be placed on the emotional experience during Web-based interventions in order to maximize reductions in smoking intentions. Going beyond mere evaluation of the effectiveness of a Web-based smoking prevention program, this study contributes to the understanding of adolescents’ psychological experience and its effect on their intention to smoke. With the results of this study, researchers can work to (1) enhance the experience of interactivity and entertainment and (2) amplify concepts of media effects (eg, presence and emotional involvement) in order to better reach health behavior outcomes.
Clinicaltrials.gov NCT02469779; https://clinicaltrials.gov/ct2/show/NCT02469779 (Archived by WebCite at http://www.webcitation.org/6nxyZVOf0)
Tobacco smoking remains the most preventable cause of death in the world [
While several Web-based programs have shown success in delaying smoking initiation among adolescents [
To fill in these gaps, it is important to better understand the processes by which Web-based interventions work to reach health outcomes. Going beyond a mere evaluation that explains “whether” a program works, it is crucial to investigate “how” a program works to be successful. This is done by studying the underlying mechanism that delineates one’s experience of a Web-based intervention.
As a result, in this study, we aimed to test associations that link participation in a Web-based program to changes in a health outcome. We achieved this aim in the context of a Web-based intervention called
With a careful design of interactive and entertaining features, users’ experience of interactivity and entertainment becomes crucial for the success of Web-based programs [
The effect of the experience of interactivity and entertainment on health outcomes is supported by the experiential learning theory (ELT) [
Supportive of the ELT and E-ELM, empirical findings show that users’ experience of interactivity and entertainment gradually drives them toward healthy outcomes. Perceived interactivity [
Although such associations have been examined each on its own, they remain fragmented. As a result, through a short-term randomized controlled trial with ASPIRE, this study developed a user experience model and statistically validated it using path analysis (
Conceptual model tested in this study. ASPIRE: A Smoking Prevention Interactive Experience.
In order to examine the net benefit of interactivity and entertainment in ASPIRE, this study was conducted post hoc, using data from a 2-arm single-blinded randomized controlled trial with assessments at baseline and immediate follow-up (time × condition). This trial, called ASPIRE Reactions, was conducted in 2014. It is registered at the ClinicalTrials.gov registry (identifier: NCT02469779). Its components adhere to the CONSORT (Consolidated Standards of Reporting Trials) and CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth) guidelines [
The trial involved 2 conditions: (1) the ASPIRE program and (2) a control condition without interactivity or entertainment (ASPIRE-control). Study manipulations are presented in
Manipulations for study conditions.
Main factor and elements | ASPIREa | ASPIRE-control | |
Two-way communication | Yes | No | |
Control over platform | Yes | No | |
Clicking behavior | Yes | No | |
Virtual environments | Yes | No | |
Narrative or storytelling | Yes | No | |
Music; sound effects | Yes | No | |
Virtual characters or avatars | Yes | No | |
Channel | Multimedia (video, audio, and text) | Text only | |
Content | Facts delivered in a personal context (age-tailored) | Facts only | |
Involvement | Emotional (peripheral processing of animation and narratives) | Cognitive (central processing of facts) |
aASPIRE: A Smoking Prevention Interactive Experience.
ASPIRE features interactivity and entertainment to engage adolescent users through text, animations, videos, and task-oriented activities (
ASPIRE-control was designed to include the same health information presented in ASPIRE but without any features of interactivity or entertainment. In order to design an appropriate control condition, we conducted qualitative content analysis of ASPIRE’s content to identify factual information about tobacco or smoking that is depicted in ASPIRE. Factual information was compiled and summarized to form a text-based document. Following this procedure, the document was fragmented to form a series of paragraphs. The paragraphs presented information in the same order as presented in ASPIRE. In order to control for exposure to online health information, the text was introduced in a mock website that had the same background design as the actual ASPIRE intervention.
A total of 4 after-school programs located in Medicaid-eligible districts of Houston, Texas, were randomly selected for participant recruitment, including the Boys and Girls Clubs (2 sites), the Salvation Army Boys and Girls Clubs (1 site), and the Young Men's Christian Association (YMCA; 1 site). After approval from the program directors, a verbal announcement reached 509 adolescents. Interested adolescents completed child assent and parental permission. We assessed adolescent eligibility through a screening conducted before participation. Inclusionary criteria were as follows: being of ages 12 through 18 years, being a student in a middle school or high school, and being a nonsmoker (have not smoked in the past year, not even one cigarette, cigar, or hookah).
All participants in the final sample obtained parental consent. Recruitment and data collection took a period of 4 months. The institutional review boards for human subjects research at the University of Texas MD Anderson Cancer Center and the University at Buffalo, the State University of New York approved this study.
All participants started their experience with the intervention 3 days after they completed the baseline survey. The principal investigator generated the random allocation sequence. Research assistants assigned enrolled adolescents to groups. We used concealed envelopes to conduct randomization. Participants were not told which intervention was the intervention of interest. In ASPIRE and ASPIRE-control, participants used identical computers and had private space for individual viewing and headphones for noise reduction. A research assistant was available for technical assistance and supervision. At the end of the intervention, participants completed the follow-up survey. Then, 20 randomly selected participants from the ASPIRE group participated in exit interviews (data not included in this paper).
As an ethical consideration, after follow-up assessment, the ASPIRE-control group received information about the actual ASPIRE intervention and ways to access the website. Each participant was offered a US $15 gift card for participation in the study. Participants received giveaway items such as pens, bags, and earphones to complete each survey.
All survey measures have been previously tested and validated. All measures were assessed through Web-based closed surveys, in the presence of a research assistant who was only available for technical assistance. We adhered to the Checklist for Reporting Results of Internet E-Surveys (CHERRIES;
Survey measures.
Measures | Description | Alphaa | |
Intention to smoke | Items adapted from the susceptibility to smoke scale [ |
.80 | |
Perceived interactivity | Measured with 17 items from Coursaris and Sung [ |
.94 | |
Perceived entertainment | A scale adapted from the work of Cyr and colleagues [ |
.92 | |
Perceived presence | Measured using 5-point Likert scale items such as “While using the website, I had a sense of being in the scenes” and “While using the website, I felt I was visiting the website’s world” [ |
.88 | |
Emotional involvement | Two items: The first item belongs to the emotional involvement dimension of the transportation concept [ |
.55b | |
Prior knowledge | 21 items tested knowledge about smoking consequences. Participants indicated if they believe such items are actual consequences of smoking by answering “yes” (coded 1 if correct), “no” (coded 1 if correct), or “I do not know” (always coded 0, as incorrect) [ |
- | |
Number of friends who smoke | One open-ended question: “How many of your friends smoke?” | - | |
Frequency of Internet use | One open-ended question: “How many hours per day do you spend on the internet?” | - | |
Number of school detentions | One open-ended question: “How many detentions or suspensions have you received at school?” | - | |
School grades | Total grade at school, based on grade point average. Answer choices were A, B, C, D, or F (coded 1, 2, 3, 4, and 5, respectively). | - | |
Perceived credibility | Two items such as “In your opinion, how believable was the information presented in ASPIRE?” [ |
aReliability coefficients with Cronbach alpha were calculated from posttest data, with the exception of measures with data collected at baseline only.
bIndicates Pearson correlation between 2 items, instead of Cronbach alpha.
We conducted power analysis for sample size calculation. In order to conduct mixed-effects models using a target power of 0.95 and an effect size of 0.20 to predict intention to smoke [
Statistical analyses were conducted using Stata version 14 (StataCorp LP). Analyses of variance (ANOVAs) and chi-square tests were conducted to capture any baseline differences between the 2 conditions with respect to covariates (eg, demographic characteristics, grades at school, number of detentions, and frequency of Internet use). Then, manipulation checks were conducted to ensure that both conditions are found to provide credible health information. Bonferroni adjustment was performed to guard against type I error in the repeated ANOVAs [
To test hypothesis 1, one-way ANOVA was conducted to assess whether using ASPIRE is related to perceived interactivity and perceived entertainment. For hypothesis 2, a repeated-measures mixed-effects model was conducted to test change in intention to smoke, in a 2 (condition) × 2 (time) design. ASPIRE effects on outcome trajectories over time were measured by the condition × time interaction term. To test hypothesis 3 on the effects of perceived interactivity and perceived entertainment, 3 repeated-measures mixed-effects models were conducted with intention to smoke as the outcome variable. Model 1 tested perceived interactivity alone, model 2 tested perceived entertainment alone, and model 3 included both as independent variables. For all models, multicollinearity was tested and the Huber-White sandwich estimator was used to correct all variance estimates for heteroskedasticity [
Hypotheses 4 through 7 were first examined using mixed-effects models and multiple regression analyses that adjusted for effects of potential confounders, as covariates. These adjustments did not alter primary conclusions (data not shown). To confirm hypotheses 1 through 3 and test for hypotheses 4 through 7 under one model (
Characteristics of study participants.
Characteristics | ASPIREa (n=50) |
ASPIRE-control (n=51) |
Total sample (N=101) |
||
12-13 | 26 (52) | 38 (74.5) | 64 (63.2) | .05 | |
14-15 | 18 (36) | 11 (21.5) | 29 (28.7) | ||
16-17 | 6 (12) | 2 (3.9) | 8 (7.9) | ||
Male | 31 (60.7) | 27 (52.9) | 58 (56.8) | .42 | |
Female | 20 (39.2) | 24 (47.0) | 44 (43.5) | ||
Hispanic or African American | 39 (78) | 47 (92.1) | 86 (85.1) | .05 | |
Non-Hispanic, non–African American | 11 (22) | 4 (7.8) | 15 (14.8) | ||
High school or less | 16 (32.6) | 23 (45.1) | 39 (39.0) | .43 | |
College or more | 33 (67.3) | 28 (54.9) | 61 (61.0) | ||
High school or less | 23 (46.9) | 29 (61.7) | 52 (54.1) | .27 | |
College or more | 26 (53.0) | 18 (38.3) | 44 (45.8) | ||
High school or less | 9 (24.3) | 12 (46.1) | 21 (33.3) | .16 | |
College or more | 28 (75.6) | 14 (53.8) | 42 (66.6) | ||
A | 37 (72.5) | 31 (60.7) | 68 (66.6) | .49 | |
B | 12 (23.5) | 17 (33.3) | 29 (28.4) | ||
C | 2 (3.9) | 2 (3.9) | 4 (3.9) | ||
D | 0 (0.0) | 1 (1.9) | 1 (0.9) | ||
Number of school detentions, mean (SD) | 1.52 (3.95) | 1.69 (2.65) | 1.61 (3.34) | .80 | |
Number of friends who smoke, mean (SD) | 1.06 (1.98) | 2.82 (4.62) | 1.96 (3.67) | .02 | |
Prior knowledge of smoking effects, mean (SD) | 14.04 (3.80) | 12.94 (4.03) | 13.49 (3.94) | .16 | |
Prior intention to smoke, mean (SD) | 1.43 (0.65) | 1.56 (0.72) | 1.50 (0.68) | .36 | |
Frequency of Internet use, mean (SD) | 3.77 (3.48) | 5.01 (3.83) | 4.40 (3.70) | .10 |
aASPIRE: A Smoking Prevention Interactive Experience.
bSignificance testing with chi-square test for the categorical variables (ie, age, gender, race or ethnicity, educational level, and school grades) and analysis of variance for the continuous variables. Missing values are not presented in this table.
A total of 110 adolescents agreed to participate. We excluded 9 adolescents who did not meet the adolescent age criterion (ages 12 through 18 years). A total of 101 participants took the baseline survey and were randomized to 1 of the 2 conditions. All 101 participants went through ASPIRE and ASPIRE-control as prescribed and completed all sessions. Then, 81.20% (82/101) continued until follow-up (81.2% completion rate;
There was no significant difference between participants who did and those who did not continue to follow-up with respect to baseline intention to smoke (
CONSORT (Consolidated Standards of Reporting Trials) flow diagram. The exit interview data are not included in this paper. ASPIRE: A Smoking Prevention Interactive Experience.
We checked to make sure that the conditions present credible health messages. As expected, there was no significant difference between the conditions with regard to perceived credibility of message content,
There was a marginal difference between the groups with respect to age, ethnicity, and number of friends who smoke (
The results indicate that the 2 groups did not differ with respect to the frequency of Internet use in hours per day (
To check for potential demographic confounders, we determined whether intervention effects varied by demographic characteristics, using moderation analysis with mixed-effects models. Overall, the results failed to identify differential effects as a function of age (
To test hypothesis 1, we checked to see if the manipulation of interactivity and entertainment has led to an experience of interactivity and entertainment among adolescent users. There was a significant difference between the ASPIRE group and the ASPIRE-control group with respect to perceived interactivity (
A mixed-effects model predicting intention to smoke and controlling for confounders showed support for hypothesis 2 (
Change in intention to smoke over time for ASPIRE (A Smoking Prevention Interactive Experience) and ASPIRE-control.
There was a significant correlation between perceived interactivity and perceived entertainment (
Repeated-measures mixed-effects models with perceived interactivity and perceived entertainment predicting intention to smoke.
Variables | Intention to smokea | |||||
Model 1 | Model 2 | Model 3 | ||||
beta (SE)b | beta (SE) | beta (SE) | ||||
Perceived interactivityc | −0.27 (0.01) | .004 | - | -0.23 (0.01) | ||
Perceived entertainmentc | - | −0.20 (0.01) | .038 | −0.06 (0.01) | .60 | |
Condition | 0.11 (0.17) | .355 | 0.10 (0.18) | .462 | 0.16 (0.18) | .37 |
Time | 0.01 (0.06) | .797 | 0.01 (0.06) | .856 | 0.01 (0.06) | .85 |
Condition × timec | −0.22 (0.11) | .007 | −0.19 (0.11) | .007 | −0.19 (0.11) | .007 |
Intercept | 3.05 (0.81) | <.001 | 2.92 (0.83) | <.001 | 3.16 (0.89) | <.001 |
Wald chi-square | 34.32 | <.001 | 35.04 | <.001 | 36.68 | <.001 |
aIndicates dependent outcome variable.
bIndicates standardized values followed by standard error.
cIndicates variables of interest. The models control for covariates (age, gender, prior knowledge, school grades, school detentions, and number of friends who smoke), with no significant relationship between such covariates and intention to smoke.
Controlling for baseline group differences and confounders, path model results remained the same. With perceived entertainment (path model 1,
With perceived interactivity (path model 2,
Path model 3 of
Path models indicating the path from ASPIRE (A Smoking Prevention Interactive Experience) use to intention to smoke. Note. By controlling for the effect of confounders and demographic group differences at baseline, the results remained the same. CFI: comparative fit index; RMSEA: root mean square error of approximation *
This post hoc study of a randomized controlled trial examined the process by which adolescents’ experience with a Web-based smoking prevention intervention leads to a health outcome. This study was the first step to better understand the underlying mechanism of eHealth effects, with a user-centered approach. The study identified salient variables in this mechanism from the perspective of the users’ experience and then postulated and empirically tested a model that can help explain how this mechanism takes place cognitively and emotionally, to reach the intended health outcome.
In particular, while ASPIRE has previously shown success in delaying smoking initiation [
The results demonstrate the advantages of a more engaging user experience perceived as interactive and entertaining over the mere exposure to health information in conventional interventions. The 2 critical elements of the user experience manipulated in this study are interactivity and entertainment, as ASPIRE features activities with real-time feedback and entertaining videos. As expected, such manipulation predicted a perception of interactivity and entertainment. The more adolescents found ASPIRE to be interactive and entertaining, the more they were likely to show a decrease in their intention to smoke. A closer look at these findings shows that perceived interactivity in ASPIRE has a stronger relationship with the decrease in intention to smoke than perceived entertainment. This may not necessarily be due to the lack of entertainment in ASPIRE. Instead, this finding may be due to adolescents’ tendency to prefer first-hand involvement in activities and experiential learning instead of exposure to entertaining videos. As a result, Web-based smoking prevention programs that use entertainment need to concentrate their efforts on interactive elements that can boost the effect of entertainment on adolescents’ intention to smoke.
The results also suggest that adolescents tend to transition from the experience of entertainment and interactivity to a psychological state of reduced intention to smoke, passing through emotional involvement. Adolescents transitioned from the experience of media elements (interactivity and entertainment) to perceived presence, emotional involvement, and ultimately lower intention to smoke. This model suggests that the elements of Web-based interventions for smoking prevention (eg, interactivity and entertainment) can have specific psychological effects beyond those explained by cognitive theories (eg, the social learning theory and the health belief model). In particular, concepts such as perceived presence and emotional involvement contribute to predicting tobacco control outcomes.
Furthermore, perceived interactivity and perceived entertainment took separate paths to reach emotional involvement in ASPIRE. Perceived interactivity did not directly correlate with a state of emotional involvement in ASPIRE. Instead, perceived presence allowed for an indirect relationship between perceived interactivity and emotional involvement. The more adolescents found ASPIRE to be interactive, the more they felt present in the ASPIRE environment. Then, perceived presence drove emotional involvement in ASPIRE. On the other hand, perceived entertainment exhibited a direct relationship with emotional involvement, consistent with previous research that presents entertainment as a driver of emotions through drama [
The findings support the notion that emotional involvement is important if we are to impact adolescent intentions. Emotional involvement seems to play a role in ASPIRE by bridging the gap between experience and smoking intentions. In addition to user activities, the videos in ASPIRE can have a strong emotional influence. Such videos portray dramatic stories through testimonials from smokers facing the effects of smoking, humoristic stories about social situations and smoking outcomes, and fear appeal through the depiction of oral and maxillofacial cancer as a result of smoking. Regardless of the user experience, entertainment and interactivity seem to elicit emotions that predict a decrease in intention to smoke. This supports previous work suggesting that messages with an emotional tone have an impact on youth smoking behavior [
Some limitations for this study must be noted. Participants in both conditions were asked to sit in front of laptops and follow a stringent regimen of ASPIRE usage. During each session, they had to keep an unchanged sitting position without peer-to-peer interaction. While this procedure did not allow adolescents to behave as in a natural environment, it created a uniform Web-based experience that controls for any contamination of results that may be due to distractions during intervention use.
Although the study design involved a manipulation of ASPIRE that mechanically removed entertainment and interactivity from the intervention to create the control condition, the study design did not separate interactivity from entertainment. ASPIRE was designed in a Flash Player format, which is costly to manipulate. In addition, several activities in ASPIRE involved an amalgam between interactivity and entertainment. As a result, the separation between these features in ASPIRE is not sound because it can disrupt the overall ASPIRE experience. In the future and outside the context of ASPIRE, we plan to conduct a study that physically separates interactivity from entertainment and compares the 2 features with each other.
The findings of the transitional model must be interpreted with caution, considering that data analysis is conducted through regression and causation cannot be inferred (eg, change over time in emotional involvement). Nevertheless, path analysis supports the presence of transition that can be confirmed in the future, through a causation model.
Although the study predicted intention to smoke, it did not consider long-term opportunities for protective behaviors or the measurement of actual smoking initiation. While smoking initiation was already examined during the main randomized controlled trial for ASPIRE [
Finally, it must be noted that the opportunity to provide a deep and comprehensive analysis is limited by the relatively small sample size. Future work may consider examining how the ASPIRE experience can prevent long-term smoking initiation with a larger sample of adolescents.
Several implications for future research and practice can be envisioned. The results of this study indicate that researchers can work to enhance the experience of interactivity and entertainment in order to better reach outcomes related to nicotine and tobacco control. First, the study of intervention experience can ultimately inform new ways to improve content of Web-based smoking prevention interventions. Through a collaboration with health communication scientists, public health researchers can work to maximize emotional involvement through the elements of interactivity and entertainment. Second, the results encourage further investigation of entertainment in order to find ways to improve its impact. Knowing that interactivity plays an important role in reducing intention to smoke, intervention designers can make use of entertaining features in an interactive environment. For instance, ASPIRE researchers may introduce game-based activities into their interventions. By incorporating game elements into purely interactive activities, adolescents may be transported into a playful environment that can increase their emotional involvement [
Screenshots of the ASPIRE (A Smoking Prevention Interactive Experience) and ASPIRE-control webpages.
Layout and design of A Smoking Prevention Interactive Experience (ASPIRE).
A video of the ASPIRE (A Smoking Prevention Interactive Experience) website presenting an activity in action.
Checklist for Reporting Results of Internet E-Surveys (CHERRIES).
CONSORT eHealth V1.6.1 checklist.
analysis of variance
A Smoking Prevention Interactive Experience
coefficient of determination
comparative fit index
Checklist for Reporting Results of Internet E-Surveys
Consolidated Standards of Reporting Trials
Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth
extended-elaboration likelihood model
experiential learning theory
root mean square error of approximation
Young Men's Christian Association
We thank the advising committee from the University at Buffalo, the State University of New York (Buffalo, New York) for their contributions to the success of this trial, including Mark Frank, PhD, and Lance Rintamaki, PhD. We also thank the University of Texas MD Anderson Cancer Center (creators of ASPIRE, Houston, Texas). Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under Award Number R25CA057730 (Principal Investigator: Shine Chang, PhD) and by the Cancer Center Support Grant CA016672 (Principal Investigator: Ronald DePinho, MD). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
GEK is responsible for the design of the study; AVP and HW provided guidance on the design of the study; GEK was responsible for the data collection, acquisition, and analysis; NM and KSC participated in the data collection and implementation of the study procedure at each research site; GEK, AVP, HW, NM, KSC, and RS contributed to the conceptualization and design of the paper; GEK drafted the paper; HW, AVP, KSC, NM, and RS critically revised the paper. All authors read and approved the final version. GEK had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
None declared.