Published on 18.11.08 in Vol 10, No 5 (2008): Theme Issue: Web-Assisted Tobacco Interventions. Guest Editors: Cameron Norman, Scott McIntosh, Peter Selby
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/993, first published Nov 14, 2007.
Comparing Two Web-Based Smoking Cessation Programs: Randomized Controlled Trial
Background: Smoking cessation remains a significant public health problem. Innovative interventions that use the Internet have begun to emerge that offer great promise in reaching large numbers of participants and encouraging widespread behavior change. To date, the relatively few controlled trials of Web-based smoking cessation programs have been limited by short follow-up intervals.
Objective: We describe the 6-month follow-up results of a randomized controlled trial in which participants recruited online were randomly assigned to either a Web-based smoking cessation program (Quit Smoking Network; QSN) or a Web-based exercise enhancement program (Active Lives) adapted somewhat to encourage smoking cessation.
Methods: The study was a two-arm randomized controlled trial that compared two Web-based smoking cessation programs: (1) the QSN intervention condition presented cognitive-behavioral strategies, and (2) the Active Lives control condition provided participants with guidance in developing a physical activity program to assist them with quitting. The QSN condition provided smoking cessation information and behavior change strategies while the Active Lives condition provided participants with physical activity recommendations and goal setting. The QSN condition was designed to be more engaging (eg, it included multimedia components) and to present much greater content than is typically found in smoking cessation programs.
Results: Contrary to our hypotheses, no between-condition differences in smoking abstinence were found at 3- and 6-month follow-up assessments. While participants in the QSN intervention condition spent more time than controls visiting the online program, the median number of 1.0 visit in each condition and the substantial attrition (60.8% at the 6-month follow-up) indicate that participants were not as engaged as we had expected.
Conclusions: Contrary to our hypothesis, our test of two Web-based smoking cessation conditions, an intervention and an attention placebo control, failed to show differences at 3- and 6-month assessments. We explored possible reasons for this finding, including limited engagement of participants and simplifying program content and architecture. Future research needs to address methods to improve participant engagement in online smoking cessation programs. Possible approaches in this regard can include new informed consent procedures that better explain the roles and responsibilities of being a research participant, new program designs that add more vitality (changing content from visit to visit), and new types of reminders pushed out to participants to encourage return visits. Simplifying program content through a combination of enhanced tailoring and information architecture also merits further research attention.
J Med Internet Res 2008;10(5):e40
The importance of smoking on public health is undeniable: “Cigarette smoking remains the leading cause of preventable mortality in the United States, resulting in nearly 16 million deaths since the first Surgeon General’s report on smoking and health in 1964” . In addition to the profound deleterious impact on smokers’ health, “secondhand smoke causes premature death and disease in children and in adults who do not smoke” [ , ]. In response to the public health risks associated with smoking, numerous research projects have examined tobacco cessation approaches that are based in clinical settings as well as public health methods that permit wider dissemination [ - ].
One nascent area of development for public health smoking cessation programs involves the use of Internet-based interventions. To date, only a handful of published studies have described the efficacy/effectiveness of Web-based tobacco cessation programs. Several reports have described promising results of trials with participants from commercial or tobacco control agency Web-based programs for smokers [- ]. Other published studies have described initial feasibility studies of Web-based tobacco cessation programs [ , ]. A few randomized controlled trials (RCTs) have been conducted [ , - ], and their results have been generally encouraging [ ]. However, interpretation of the randomized trials is complicated by short follow-up intervals, for example, 6 weeks in Strecher et al [ ], 2.5 months in Etter [ ], and 3 months in Lenert et al [ ], Pike et al [ ], and Swartz et al [ ]. Additional trials with longer term follow-up assessments examining the use of Web-based interventions for smoking control are clearly warranted.
The current paper describes results of the Smokers’ Health Improvement Program (SHIP) RCT that examined the 3- and 6-month outcomes for two Web-based programs: (1) an intervention condition (Quit Smoking Network; QSN) that provided users with extensive information and behavioral strategies drawn from clinic-based and self-help smoking cessation programs, or (2) a Web-based control condition that focused on increasing physical activity (Active Lives). Currently, because grant funding has ended, neither the QSN nor Active Lives website is currently available for review. However, screenshots of the programs are included in the.
In addition to primary outcome measures of tobacco abstinence, we sought to examine the impact of condition on secondary outcomes, including participant exposure to program content, physical activity, and pharmacotherapy use. We also sought to test the putative predictors of outcome. Finally, we assessed the extent to which participants found their assigned program easy to use.
The SHIP study was a two-arm RCT that compared two fully automated, Web-based smoking cessation programs: (1) the QSN intervention condition that presented cognitive-behavioral strategies, and (2) the Active Lives control condition in which participants received guidance in developing a personal fitness program to help them quit smoking. The QSN condition provided participants with smoking cessation information and behavior change recommendations while the Active Lives condition provided participants with physical activity recommendations, monitoring, and goal setting. The QSN condition was designed to be more engaging (it included video testimonials, for example), and it presented much greater content that is more typically found in smoking cessation programs. The Active Lives program provided content more explicitly aimed at increasing physical activity as a smoking cessation approach, and, although there is some evidence that physical activity adjuncts to smoking cessation programs may be helpful (eg, ), the Active Lives condition was intended to be a credible attention placebo control condition. Follow-up assessments occurred at 3 and 6 months following program enrollment. The trial was not registered, because enrollment started in spring 2005, before trial registration became mandatory.
Enrollment and Participants
We first sought to recruit participants through large worksites in order to reach our recruitment goals and to minimize attrition. This strategy proved unsuccessful. In consultation with an Internet marketing firm, we designed and executed a purely Internet-based recruitment campaign. The campaign involved ad placement on Google and Yahoo search engines (keywords “quit smoking” and “stop smoking”) and links to their relevant affiliated sites. Clicking our ads enabled users to (1) visit our recruitment site (study description, inclusion/exclusion criteria), (2) submit answers to screening items, (3) provide their informed consent, and (4) complete the baseline assessment. This Internet marketing campaign was remarkably successful: we recruited 2318 participants in only 10 weeks at a cost of approximately US $13 per recruit. A total of 69.8% (1169/2318) came from Google, 19.9% (461/2318) from Yahoo, and the remaining 10.3% (238/2318) were recruited from word of mouth or from unknown other sources. The flow of participants across various study milestones is depicted using a CONSORT diagram in. Note that 44.3% (1028/2318) of participants completed the 3-month follow-up assessment, 39.2% (909/2318) completed the 6-month assessment, and 631 (27.2%) of the randomized sample completed both the 3- and the 6-month assessments.
Study inclusion criteria were as follows: (1) at least 18 years of age, (2) current smoker interested in quitting within the next 30 days, (3) willingness to engage in moderate physical activity, (4) access to the Internet, (5) agreement with the informed consent statement as approved by the Institutional Review Board of Oregon Research Institute, and (6) completion of both program registration and the baseline assessment. Exclusion criteria included any positive answers on the 8-item Physical Activity Readiness Questionnaire (PAR-Q), designed to identify individuals for whom physical activity might be inappropriate or who should receive medical advice concerning the type of activity most suitable for them. Example exclusion items included “Has your doctor ever said you have heart trouble?” and “Do you often feel faint or have spells of severe dizziness?” .
Screening and baseline assessments were used to measure the characteristics of participants, including age, gender, race/ethnicity, current smoking, rurality , and education. Study participants were predominantly white, urban, 30- to 50-year-old married women who had at least some college education and smoked 1-2 packs of cigarettes per day at baseline (see participant characteristics in ). No statistically significant differences were found between conditions on any participant characteristics.
|< 30||266 (23.0)||253 (21.8)||519 (22.4)|
|30-39||256 (22.1)||286 (24.7)||542 (23.4)|
|40-49||360 (31.1)||327 (28.2)||687 (29.6)|
|≥ 50||277 (23.9)||293 (25.3)||570 (24.6)|
|Female||805 (69.5)||829 (71.5)||1634 (70.5)|
|Married||697 (60.1)||730 (63.0)||1427 (61.6)|
|White||990 (86.8)||982 (86.4)||1972 (86.6)|
|Black||78 (6.8)||76 (6.7)||154 (6.8)|
|Other||72 (6.3)||79 (6.9)||151 (6.6)|
|No high school degree||79 (6.8)||80 (6.9)||159 (6.9)|
|High school graduate||302 (26.1)||276 (23.8)||578 (24.9)|
|Some college||453 (39.1)||490 (42.3)||943 (40.7)|
|College graduate||325 (28.0)||313 (27.0)||638 (27.5)|
|Rural vs urban||226 (19.9)||220 (19.5)||446 (19.7)|
|Number of cigarettes smoked/day|
|≤ 10||194 (16.8)||199 (17.2)||100 (17.0)|
|11-20||497 (42.9)||458 (39.5)||955 (41.3)|
|21-40||423 (36.5)||442 (38.2)||865 (37.4)|
|≥ 41||44 (3.8)||57 (4.9)||101 (4.4)|
Description of the Web-Based Programs
QSN Intervention Condition
The QSN condition incorporated a hybrid information architecture  in which first-time users were directed through a series of tailored Web pages (tunnel design) in order to introduce them to the key concepts and strategies of a behavioral program for quitting smoking. Once they emerged from the tunnel, users were able to choose their own path to access a broad array (using a matrix design) of additional content on quitting and maintaining nonsmoking.
Components of the smoking cessation intervention used in the study are based on Social Cognitive Theory [, ] as it has been applied to tobacco abstinence [ , ]. These components are designed to help encourage tobacco abstinence via the use of strategies that address each participant’s behavior, cognition, and environment [ , ]. This approach also builds on behavioral self-management [ , ], in which the intervention is viewed as providing structure, skills, and a supporting scaffold that encourage the participant to become an active problem solver in the iterative process of trying out and then refining the use of a series of strategies as a part of a personalized plan for quitting tobacco.
Key content modules focused on getting ready, developing a personal quitting plan, setting a personal quit date, using pharmacotherapy products including nicotine replacement therapy, avoiding and altering trigger situations, using substitutes, managing thoughts, using strategies to manage mood, and obtaining support from a peer-to-peer Web forum as well as a professionally moderated “Ask an Expert” forum. The program also offered an extensive library of additional content. Because users were required to log in to the website using their unique username and password, we were able to tailor portions of the program content to each participant’s smoking/nonsmoking status (checked at the start of each session) and to display online prompts recommending the review of program content that a participant had not yet explored.
Active Lives Control Condition
Participants assigned to the Active Lives control condition accessed a Web-based program designed to encourage them to engage in a personalized fitness program that would help them quit smoking. The Active Lives program was based primarily on Bandura’s Social Cognitive Theory [, ], research on interventions to promote physical activity [ , ], and our earlier research on an online diabetes self-management intervention [ ]. The program guided each participant through a multi-step plan that included a motivational component (exploration of the benefits of physical activity and a clarification of personal goals and barriers), a behavioral action plan with extensive tracking features (eg, weekly activity schedules personalized to each participant’s schedule and types of activities), additional online resources (articles and tips sheets), and access to a Web forum for peer support.
Method of Assessment (Online and Phone)
We attempted to collect all participant assessments via the Internet. Participants were sent an email reminder 3 days prior to an assessment and on the due date of that assessment. Participants who failed to complete their online assessment within 1 week were sent an additional email reminder. If participants had not completed their online assessment within a 2-week period, then a project research assistant initiated a process to complete assessments by phone.
Primary Outcome Measures: Smoking Cessation
Following the recommendation of the Society of Research on Nicotine and Tobacco , we assessed point prevalence smoking/nonsmoking status by asking “Have you smoked any cigarettes in the last week, even a puff?” Each participant’s smoking abstinence was measured at 3 and 6 months post-enrollment. In addition, we examined repeated point prevalence nonsmoking at both the 3- and 6-month assessments. Because our intervention was accomplished entirely online without any personal contact between participant and researchers, we concluded that it was impractical to obtain biochemical validation of self-reported abstinence—a decision consistent with both the recommendations of Glasgow et al [ ] for low-intensity intervention trials as well as with many of the published trials of Web-based smoking cessation programs (eg, [ , , ]).
In addition to baseline demographic and smoking data (see), the set of putative predictors we planned to examine included self-efficacy, dependence, support for quitting, smoking among friends and family members, depression, and prior quit attempts. Tests of putative predictors help to establish the veracity of the dataset because they broaden the knowledge base and enable comparisons with other similar findings in the literature. Reporting predictors could help inform future intervention design in showing characteristics of those participants for whom the programs seemed most efficacious. Examples of studies that have examined predictors of outcome using participants who had participated in a tobacco cessation RCT include those performed by Oregon Research Institute researchers [ , , ] and other research teams (eg, [ , , - ]).
Confidence in accomplishing various facets of quitting smoking was assessed at baseline using a 5-point Likert scale (1 = not at all confident to 5 = very confident). Items included the following: “If you decided to quit smoking, how confident are you that you could quit?”, “If you decided to quit smoking, how confident are you that you will not be smoking a year from now?”, “How confident are you that you can resist smoking when you are feeling bored or restless?”, “How confident are you that you can resist smoking when you are angry, frustrated, or tense?”, “How confident are you that you can resist smoking when you drink alcohol?”, and “How confident are you that you can resist smoking when you are around others who are also using it?
Nicotine dependence was measured at baseline using an item excerpted from a scale developed by Piper et al : “How strong are your urges when you first wake up in the morning?” This was assessed with a 7-point Likert scale (1 = not strong at all to 7 = extremely strong). In addition, participants were asked at baseline to answer the following question using one of eight answer options (1-5, 6-10, 11-15, 16-20, 21-25, 26-30, 31-40, 40 or more): “On average, how many cigarettes do you smoke each day?”
Support for Quitting
Participants were asked at baseline to rate the expected support for quitting: “If you decided to quit smoking, how supportive would the person you’re closest to be of your efforts to stop smoking?” (1 = not at all supportive to 7 = very supportive).
Smokers Among Friends and Family
Participants were asked to answer two items recommended by Piper et al  using a 7-point Likert scale (1 = not true of me at all to 7 = extremely true of me): “A lot of my friends or family smoke” and “Most of my friends and acquaintances smoke.”
The baseline assessment asked participants to answer two dichotomous (yes/no) items that measured depression: “In the past year, have you had 2 weeks or more during which you felt sad, blue, or depressed; or when you lost all interest or pleasure in things that you usually cared about?” and “Have you had 2 years or more in your life when you felt depressed or sad most days, even if you felt okay sometimes?”
Participants were asked to answer the question “In the past year, how many times have you made a serious attempt to quit?” using five answer options: 0, 1, 2, 3, or ≥ 4 attempts.
Secondary Outcome Measures
The extent to which participants accessed their assigned Web-based program was measured unobtrusively using a combination of database tracking and Web server log analysis  to determine both number of visits (sessions) and duration of visits. We also created a composite measure of exposure (mean of standard scores for the number of visits and total time spent across all visits). We examined the pattern of declining participant visits to the Web-based programs following enrollment [ ] by calculating each participant’s final visit date to view program content (visits associated with completing online assessments were not included). If a participant viewed program content only one time and it occurred on his/her enrollment date, then he/she would be assigned 0 days (last visit occurred zero days since the day of enrollment).
Two items from the Behavioral Risk Factor Surveillance System (BRFSS)  were used to measure whether participants engaged in vigorous or moderate levels of continuous activity: “In a usual week, do you do vigorous activities for at least 10 minutes at a time, such as running, aerobics, heavy yard work, or anything else that causes large increases in breathing or heart rate?” and “In a usual week, do you do moderate activities for at least 10 minutes at a time, such as brisk walking, bicycling, vacuuming, gardening, or anything else that causes some increase in breathing to heart rate?” Endorsement of these items was followed by the following question: “How many days per week do you do vigorous [moderate] activities?”
Participants in both conditions were asked the following question on the 3- and 6-month assessments: “Which of the following products or methods have you tried in the last 3 months?” Answer options included nicotine gum, nicotine patches, nicotine lozenges, nicotine spray, nicotine inhaler, other nicotine replacement product, and Zyban (bupropion). Two composite scores were derived for each participant: the sum of pharmacotherapy products used (from 0 to 6) and a yes/no score for any pharmacotherapy use.
At each of the 3- and 6-month assessments, participants were asked to answer “How easy was it to use the QSN/Active Lives program?” using a 5-point Likert scale (1 = not at all; 3 = somewhat; 5 = very).
Logistic Regression and Survival Analysis
Putative predictors were tested using two complementary steps. First, we used binary logistic regression models that incorporated treatment condition as well as the interaction of the condition with each variable in order to identify any differential effects of the intervention on the prediction of smoking abstinence . Second, significant predictors were then tested in a multivariate binary logistic regression using backwards elimination. Following the approach we used in another Web-based tobacco cessation program [ ], we used Kaplan-Meier survival analyses [ , ] to examine the pattern of last visits to the Web-based program using the number of days post-enrollment as our unit of time. All analyses were conducted using SPSS statistical software, version 15 (SPSS Inc, Chicago, IL, USA).
Complete Case vs Intent-to-Treat Analyses
Given that only 39.2% (909/2318) of participants responded to the 6-month follow-up assessment, we decided not to use complex imputation methods . Instead, we used two complementary approaches: (1) complete case analysis limited to data obtained from participants who responded to a follow-up assessment, and (2) an intent-to-treat analysis that used the original sample as the denominator and a simple imputation method in which all participants who did not complete an assessment were considered to be still smoking (missing = smoking).
Assessment Completion / Participant Attrition
Of the 1028 participants who completed the 3-month assessment, 315 (30.6%) did so online and 713 (69.4%) were contacted by phone. Of the 909 participants who completed the 6-month assessment, 161 (17.7%) completed it online and 748 (82.3%) completed a phone assessment. The two conditions did not differ in terms of the proportion of online assessments at 3 months, but QSN had more online 6-month assessments than the Active Lives condition: 21.4% (96/448) vs 14.1% (65/461); χ21 = 8.34, N = 909, P = .004.
We used multivariate logistic regression on complete cases to examine possible baseline predictors using two dependent variables: (1) completion of the 6-month assessment and (2) completion of both the 3- and 6-month assessments. Positive predictors of completing the 6-month assessment included age (adjusted odds ratio [OR] = 1.33, 95% confidence interval [CI] = 1.23-1.44, P < .001), education level (OR = 1.18, CI = 1.07-1.29, P = .001), and confidence in being able to quit for one year (OR = 1.08, CI = 1.00-1.15, P = .04). Positive predictors of completing both the 3- and 6-month assessments included age (OR = 1.40, CI = 1.30-1.51, P < .001), marital status (OR = 1.13, CI = 1.05-1.48, P = .01), education level (OR = 1.13, CI = 1.03-1.24, P = .01), and confidence in being able to quit for one year (OR = 1.10, CI = 1.03-1.18, P = .008). None of the condition by predictor tests reached statistical significance.
Primary Outcome: Tobacco Cessation
Self-reported smoking abstinence was examined by condition using complete case and intent-to-treat analyses at 3 months, 6 months, and also for both the 3- and 6-month follow-up assessments (see). Results for the QSN and Active Lives conditions were remarkably similar.
Binary logistic regression tests (complete case) failed to uncover any significant differences in smoking abstinence between the QSN and Active Lives conditions when we considered assessments at 6 months or when we considered nonsmoking at both 3 and 6 months (see). Intent-to-treat (missing = smoking) analyses showed similar nonsignificant between-group differences.
|3 and 6 Months|
|QSN||103/524 (19.7)||112/448 (25.0)||45/314 (14.3)|
|Active Lives||99/504 (19.6)||120/461 (26.0)||44/317 (13.9)|
|QSN||103/1159 (8.9)||112/1159 (9.7)||45/1159 (3.9)|
|Active Lives||99/1159 (8.5)||120/1159 (10.4)||44/1159 (3.8)|
|Both 3 and 6 Months|
Predictors of Outcome
None of the interactions between putative predictors and treatment condition were significant. Results of multivariate binary logistic tests performed on smoking abstinence for all participants (collapsed across condition) at 3 months, 6 months, and both 3 and 6 months are presented in. Education emerged as a significant positive predictor of smoking abstinence in all three cases. Baseline smoking rate was negatively related to smoking abstinence on each of the single follow-up assessments. Expected support for quitting had a positive relation with smoking abstinence at the 3-month assessment and the combined 3- and 6-month assessment.
|Both 3 and 6 Months|
|Serious quit attempts||−.121||.81||.67||.98||.030|
The frequency and duration of each participant’s visits to the Web-based program (not including visits to complete online assessment) were measured unobtrusively in Web server logs and using database tracking methods (seeand ). Compared to the Active Lives condition, participants in the QSN condition averaged more visits (2.14 visits, SD = 3.66 vs 1.74 visits, SD = 2.43; unequal variance t2012.03 = −3.11, P = .002). Analysis of data on duration of program usage by condition ( ) revealed that the QSN condition was notably less kurtotic and skewed than the Active Lives condition (kurtosis = 21.79 vs 181.70; skewness = 3.53 vs 10.38). Analysis of these data revealed that participants in the QSN condition spent significantly more time visiting the program (18.04 minutes, SD = 22.18 vs 14.02 minutes, SD = 17.09; unequal variance t2174.56 = −4.02, P < .001).
|Number of Visits||Duration of Visits (minutes)|
|QSN||2.14a||3.66||1.00||1 (1-2)||18.04b||22.18||10.00||19 (5-24)|
|Active Lives||1.74||2.43||1.00||1 (1-2)||14.02||17.09||11.00||11 (6-17)|
aBetween-condition comparison: P = .001.
bBetween-condition comparison: P < .001.
We examined the pattern of participant exposure over time by condition. For purposes of this analysis, we defined exposure as the number of days elapsed between each participant’s date of randomization/enrollment and the date of his/her last website visit to view program content. We applied the Kaplan-Meier survival analysis to these data, which allowed us to examine the timing of the last visit by condition (see). By definition, all participants had a last visit since all participants stopped visiting at some point following enrollment. Note that shows a steep downward slope in last program visits soon after program enrollment, indicating that most participants stopped visiting the program soon after they started. There were notable drops in subsequent participation at times that corresponded with the follow-up assessments. While QSN had somewhat longer estimated survival time (mean = 36.71 days, SE = 2.18) than the Active Lives condition (mean = 30.86, SE = 2.02), the Kaplan-Meier survival tests revealed that the overall trajectory of these post-enrollment program visit curves did not significantly differ by condition: Breslow (2.30, P = .13) and Log Rank Mantel-Cox (1.97, P = .16).
Binary logistic regression analyses failed to reveal any significant dose-response effects using the composite exposure score (incorporating both number of visits and total usage duration) and smoking abstinence at both 3 and 6 months within and combined for conditions (complete case analyses).
At the 6-month assessment, remarkably similar proportions of participants in the QSN and Active Lives conditions reported that they engaged in vigorous physical activity—40.2% (173/430) vs 38.0% (163/429), respectively—and moderate physical activity—76.7% (332/433) vs 79.4% (344/433), respectively. No significant group differences were found in the reported number of days the participant had engaged in at least 10 minutes of moderate or vigorous physical activity (group t values ranged from .13-1.33). Number of days of vigorous activity tended to increase slightly across the baseline, 3-month, and 6-month assessments for both the QSN (mean = 3.42, SD = 1.52; mean = 3.74, SD = 1.51; mean = 3.69, SD = 1.46; respectively) and Active Lives conditions (mean = 3.47, SD = 1.62; mean = 3.51, SD = 1.65; mean = 3.85, SD = 1.49; respectively). The days of moderate activity were slightly higher for each group and also increased slightly across the baseline, 3-month, and 6-month assessments for both the QSN (mean = 3.93, SD = 1.75; mean = 4.40, SD = 1.91; mean = 4.44, SD = 1.83; respectively) and Active Lives conditions (mean = 4.01, SD = 1.76; mean = 4.42, SD = 1.80; mean = 4.33, SD = 1.88; respectively).
Between-group differences in terms of the number of pharmacotherapy products participants reported using were not significant at the 3-month assessment (mean = .68, SD = .86 vs mean = .60, SD = .83; unequal variance t1026.97 = −1.54, P = .12), but participants in the QSN condition reported using significantly more pharmacotherapy products than those in the Active Lives condition at the 6-month assessment (mean = .06, SD = .82 vs mean = .55, SD =.71; unequal variance t880.06 = −1.76, P = .04, 1-tailed).
We also examined pharmacotherapy use using a composite variable that measured any use (yes/no). At the 3-month assessment, significantly more QSN participants reported using any pharmacotherapy than did the Active Lives participants: 54.2% (262/524) vs 45.8% (221/504); χ21 = 3.90, N = 1028, P = .048. However, no group-wise differences emerged on this dimension at the 6-month assessment, with pharmacotherapy use reported by 47.3% (212/448) of QSN participants compared with 43.8% (202/461) of Active Lives participants.
We analyzed ratings of program ease of use obtained at the 3-month assessment from 67.0% (351/524) of QSN participants and 72.6% (366/504) of Active Lives participants. Results favored QSN over the Active Lives control condition (mean = 3.85, SD = 1.28 vs mean = 3.65, SD = 1.36; with a rating of 3 = somewhat; unequal variance t714.79 =−2.05, P = .04).
Usability ratings obtained at the 6-month assessment from 79.0% (354/448) of QSN participants and 60.1% (277/461) of Active Lives participants showed a similar relative advantage for the QSN condition: mean = 4.10, SD = 1.21 vs mean = 3.70, SD = 1.35; unequal variance t629 = −3.91, P < .001.
It is important to note that participants were also asked to rate how helpful they found specific program areas (eg, library of materials and the support group area), but we chose not to report these data because very few individuals provided data on these items and a number of those participants who did provide such ratings did not, in fact, visit the program area based upon our unobtrusive tracking of their use of their assigned website.
The outcome results did not support our hypothesis that the QSN online smoking cessation intervention would be more effective than a credible control condition. The unremarkable impact of the QSN condition relative to the Active Lives condition is particularly surprising given that the Active Lives control condition presented very few strategies for quitting smoking since it was largely focused on helping participants improve their personal level of physical activity. The absolute level of nonsmoking at 6 months—less than 4% abstinence using intent-to-treat analysis—is less than results for other Web-based smoking cessation programs reported by Muñoz et al (5.6% to 26.0% abstinence at 6 months) . However, with respect to the nonsignificant finding between conditions, the engagement of physical activity among the Active Lives participants was noteworthy. These data reflect their relatively high level of adherence to the recommended behavior change goal. Almost 80% of participants reported that they were engaged in moderate physical activity at the 6-month follow-up assessment. This implies that our control group was actively following the recommendations suggesting the importance of engaging in physical activity in aiding quit attempts and might explain, in part, the relative lack of difference in findings between the groups on quit success.
There are noteworthy strengths and limitations to consider when interpreting these findings. Strengths include the large sample of 2318 participants, the fact that the Web-based programs were fully automated to assure high fidelity of content delivery, and that the study used a RCT methodology.
One important limitation involved participant attrition (failure to complete follow-up assessments). At the 3-month follow-up, we experienced a 55.6% (1289/2318) attrition rate, which is larger than the 44% attrition rate reported by Swartz et al  but comparable to the levels reported for other Web-based smoking cessation programs: 57.2% by Stoddard et al [ ], 62.5% by Strecher et al [ ], 70.7% by Cobb et al [ ], and 64.6% by Etter [ ]. At the 6-month follow-up, we experienced a 60.8% (1409/2318) attrition rate, which is roughly comparable to the attrition at 6 months reported by Muñoz et al [ ] for four studies of English- and Spanish-language Web-based smoking cessation programs: 73.9% (2051/2774), 69.9% (491/702), 65.7% (184/280), and 35.4% (102/288).
Although mean program exposure measures (especially duration) favored the QSN program when compared with the Active Lives control condition, the extent of these observed differences was not as large as had been expected. Nor did we find a dose-response relationship between program exposure and smoking abstinence at follow-up. Using median data, participants in the current study used the program for a single visit that lasted about 10 minutes.
Differences in study design among published studies of Web-based smoking cessation interventions and the nascent stage of the science make it difficult to generalize our program exposure results to the available body of research in this area. For example, some researchers have not reported exposure data [, , ], while others [ ] reported number of pages viewed and average visit duration but not a precise calculation of exposure. The pattern of program visits (mean = 2.14, median = 1.0) observed for both conditions in this study is lower than the findings reported by Swartz et al [ ] and Lenert et al [ ] but roughly consistent with results of other online cessation programs reported by Muñoz et al [ ]. In their comparison of five different Web-based smoking cessation interventions, Pike et al [ ] noted that the two websites with the highest “utilization rates” had 9.7 and 6.0 visits per participant, while the visit rate on the remaining websites ranged from 1.8 to 2.2 visits.
It is important to note that Japuntich et al  reported much higher levels of exposure to an adjunctive Web-based intervention by participants who had multiple in-person contacts with research project staff (including a physical exam, several counseling visits, and five in-person follow-up assessments). It is reasonable to assume that there are likely to be significant differences between adjunctive use of Web-based interventions compared to fully automatic Web-based interventions like ours that do not involve any in-person contact and that deliver the intervention only by a rules-driven algorithm.
Nonetheless, there remains the opportunity to improve participant engagement in online smoking cessation programs. For example, future research should consider testing whether engagement might be facilitated by changing program content from visit to visit (enhancing vitality), by using more effective tailoring to improve the relevance of program content, and/or by using innovative reminders (eg, some combination of email, regular mail, text messages, e-cards) that encourage multiple program visits.
Analyses of pharmacotherapy usage were supportive of the fact that the QSN condition compared to the control condition encouraged significantly greater use of this treatment approach (more than 60% vs approximately 47%), although it is important to acknowledge that almost half of participants in the control condition used pharmacotherapy without being explicitly told to do so. Similarly, Swartz et al  reported that a majority of wait-list control participants indicated that they used pharmacotherapy products.
One interesting possibility is that our use of a physical activity control condition may have inadvertently jeopardized, to some extent, the generalizability of our results. Specifically, because of concerns about health among participants in the Active Lives condition, we excluded 61 individuals (prior to randomization) who had positive answers on the PAR-Q assessment, and an additional 48 individuals declined to participate because they did not want to increase their level of exercise activity. These 109 individuals would not have been excluded from a typical smoking cessation RCT. However, it remains for further research to determine whether these exclusion criteria may have had the effect of excluding individuals who otherwise would have been successful quitters using the treatment condition.
We considered possible control/comparison conditions for the present study. A no-treatment or waiting list control group is often recommended in order to reduce the likelihood of a type II (false negative) error when an intervention is expected to produce small effects. But we concluded that offering participants no treatment or delayed treatment would have been unhelpful because waiting list controls provide only limited information . Moreover, from a pragmatic perspective, we were concerned that individuals assigned to a waiting list condition would have little reason to remain involved and would therefore be more likely to be lost to follow-up. When using an intent-to-treat model that defines missing = smoking, such differential attrition would have biased results in favor of the intervention condition or type I error [ ]. Thus, we used a control/comparison condition that offered a face-valid, credible, Web-based intervention for smoking cessation that did not contain what have been shown to be the active ingredients of evidence-based smoking cessation intervention. It is reasonable to question whether a better choice of control condition would have been a basic or static information website similar to what has been used in other RCTs of tobacco cessation interventions (eg, [ , ]).
Our remarkably successful use of online recruitment may have resulted in recruiting smokers who were less likely (1) to remain fully involved over time in a research program and (2) to quit smoking. It was quite easy for prospective participants to enroll—we estimate that it would require less than 15 minutes from clicking on a link in a Google ad listing to completing the screening, online consent, registration, and baseline assessment. The absence of measures describing ease of enrollment (either self-report or using a measure of convenience like elapsed time) makes it impossible for us to discuss further the extent to which the present study may have had easier enrollment than other Web-based behavioral interventions. Adding more barriers/hurdles to this process would very possibly have increased engagement and reduced program attrition because only motivated individuals would have been able to participate [, ]. But, while culling out less motivated individuals, this approach might spuriously inflate absolute effect sizes while reducing external generalizability [ ]. It remains for future research to determine the extent to which open online enrollment with few barriers for entry results in greater attrition.
Another likely limitation of the QSN intervention is that it may have been too expansive; that is, it may have offered too much (sometimes redundant) content that forced users to navigate through a complex information architecture, which reduced utility and encouraged attrition. We obtained ratings of program usability at follow-up, but, of course, interpretation of such data is constrained by the fact that they are drawn from a minority of original participants who completed follow-up assessments. Content duplication and/or complex intervention design can erode therapeutic impact—more is not necessarily better . Indeed, the likely mechanisms of change may not be best described by a linear dose-response relationship [ , ].
In many ways, the results of the current research underscore the complexity of developing and then evaluating Web-based interventions for smoking cessation. We used an intervention and an attention placebo control that recommended physical activity as a smoking cessation tool. Our initial plan was to recruit through worksites as had been done with success in other Web-based smoking cessation research . Because we found that worksite recruitment produced little in the way of results, we turned to online recruitment tied to popular Web search engines, particularly Google. Using this approach, we were able to rapidly randomize over 2300 participants to our RCT of two Web-based smoking cessation programs that had been carefully crafted to provide users with the content they needed in an online context that they would find interesting and engaging.
In summary, the results of this Web-based smoking cessation intervention RCT failed to confirm our hypotheses. Negative findings play an essential role in the development of science  and are particularly illuminating with regard to shaping a reasoned appreciation for the complexity of creating, delivering, and evaluating online health behavior change interventions. As we perform empirical tests of evolving interventions in this nascent field, we need to learn from both negative and positive outcomes as we strive to understand the factors that are associated with participant recruitment, program exposure, content tailoring, and participant attrition.
Thanks are extended to John Noell and Edward Feil who played key roles in the early development phases of this project. In addition, we acknowledge the contribution of Ron Williams who had primary responsibilities for programming and website development. This research was supported by grant R01-CA79946 from the National Cancer Institute. Because their hosting was supported solely by grant research funds that have now been exhausted, the Web-based interventions described in this report are no longer accessible on the Internet.
Conflicts of Interest
PowerPoint presentation of program screenshotsPPT file (Microsoft Powerpoint File), 3.1 MB
- ; US Department of Health and Human Services. The Health Consequences of Smoking: A Report of the Surgeon General. Atlanta, GA: Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health; 2004.
- ; US Department of Health and Human Services. The Health Consequences of Involuntary Exposure to Tobacco Smoke. Atlanta, GA: Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health; 2006.
- Beatty SE, Kahle LR, Homer P, Misra S. Alternative measurement approaches to consumer values: the list of values and Rokeach value survey. Psychol Market 1985:181-200. [CrossRef]
- Lichtenstein E. From rapid smoking to the Internet: five decades of cessation research. Nicotine Tob Res 2002 May;4(2):139-145. [Medline] [CrossRef]
- Fiore MC, Jean CR, Baker TB, Bailey WC, Cohen SJ, Dorfman SF, et al. Treating Tobacco Use and Dependence: 2008 Update. Washington, DC: US Department of Health and Human Services, Public Health Service; 2008. URL: http://www.surgeongeneral.gov/tobacco/treating_tobacco_use08.pdf [accessed 2008-05-27] [WebCite Cache]
- Glasgow RE, McKay HG, Piette JD, Reynolds KD. The RE-AIM framework for evaluating interventions: what can it tell us about approaches to chronic illness management? Patient Educ Couns 2001 Aug;44(2):119-127. [Medline] [CrossRef]
- Cobb NK, Graham AL, Bock BC, Papandonatos G, Abrams DB. Initial evaluation of a real-world Internet smoking cessation system. Nicotine Tob Res 2005 Apr;7(2):207-216. [Medline] [CrossRef]
- Etter JF. Comparing the efficacy of two Internet-based, computer-tailored smoking cessation programs: a randomized trial. J Med Internet Res 2005;7(1):e2 [FREE Full text] [Medline] [CrossRef]
- Pike KJ, Rabius V, McAlister A, Geiger A. American Cancer Society's QuitLink: randomized trial of Internet assistance. Nicotine Tob Res 2007 Mar;9(3):415-420. [Medline] [CrossRef]
- Zbikowski SM, Hapgood J, Barnwell SS, McAfee T. Phone and Web-based tobacco cessation treatment: real-world utilization patterns and outcomes for 11,000 tobacco users. J Med Internet Res. In press .
- Stoddard J, Delucchi K, Muñoz R, Collins N, Stable EP, Augustson E, et al. Smoking cessation research via the internet: a feasibility study. J Health Commun 2005;10(1):27-41. [Medline] [CrossRef]
- Feil EG, Noell J, Lichtenstein E, Boles SM, McKay HG. Evaluation of an Internet-based smoking cessation program: lessons learned from a pilot study. Nicotine Tob Res 2003 Apr;5(2):189-194. [Medline] [CrossRef]
- Muñoz RF, Lenert LL, Delucchi K, Stoddard J, Perez JE, Penilla C, et al. Toward evidence-based Internet interventions: A Spanish/English Web site for international smoking cessation trials. Nicotine Tob Res 2006 Feb;8(1):77-87. [Medline] [CrossRef]
- Lenert L, Muñoz RF, Stoddard J, Delucchi K, Bansod A, Skoczen S, et al. Design and pilot evaluation of an internet smoking cessation program. J Am Med Inform Assoc 2003;10(1):16-20 [FREE Full text] [Medline] [CrossRef]
- Lenert L, Muñoz RF, Perez JE, Bansod A. Automated e-mail messaging as a tool for improving quit rates in an internet smoking cessation intervention. J Am Med Inform Assoc 2004;11(4):235-240 [FREE Full text] [Medline] [CrossRef]
- Strecher VJ, Shiffman S, West R. Randomized controlled trial of a web-based computer-tailored smoking cessation program as a supplement to nicotine patch therapy. Addiction 2005 May;100(5):682-688. [Medline] [CrossRef]
- Strecher VJ, McClure JB, Alexander GL, Chakraborty B, Nair VN, Konkel JM, et al. Web-based smoking-cessation programs: results of a randomized trial. Am J Prev Med 2008 May;34(5):373-381. [Medline] [CrossRef]
- Swartz LHG, Noell JW, Schroeder SW, Ary DV. A randomised control study of a fully automated internet based smoking cessation programme. Tob Control 2006 Feb;15(1):7-12 [FREE Full text] [Medline] [CrossRef]
- Severson HH, Gordon JS, Danaher BG, Akers L. ChewFree.com: evaluation of a Web-based cessation program for smokeless tobacco users. Nicotine Tob Res 2008 Feb;10(2):381-391. [Medline] [CrossRef]
- Walters ST, Wright JA, Shegog R. A review of computer and Internet-based interventions for smoking behavior. Addict Behav 2006 Feb;31(2):264-277. [Medline] [CrossRef]
- Marcus BH, Lewis BA, Hogan J, King TK, Albrecht AE, Bock B, et al. The efficacy of moderate-intensity exercise as an aid for smoking cessation in women: a randomized controlled trial. Nicotine Tob Res 2005 Dec;7(6):871-880. [Medline] [CrossRef]
- Shephard RJ. PAR-Q, Canadian Home Fitness Test and exercise screening alternatives. Sports Med 1988 Mar;5(3):185-195. [Medline] [CrossRef]
- Danaher BG, Hart LG, McKay HG, Severson HH. Measuring participant rurality in Web-based interventions. BMC Public Health 2007;7:228 [FREE Full text] [Medline] [CrossRef]
- Danaher BG, McKay HG, Seeley JR. The information architecture of behavior change websites. J Med Internet Res 2005;7(2):e12 [FREE Full text] [Medline] [CrossRef]
- Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev 1977 Mar;84(2):191-215. [Medline] [CrossRef]
- Bandura A. Health promotion by social cognitive means. Health Educ Behav 2004 Apr;31(2):143-164. [Medline] [CrossRef]
- Brandon TH, Herzog TA, Irvin JE, Gwaltney CJ. Cognitive and social learning models of drug dependence: implications for the assessment of tobacco dependence in adolescents. Addiction 2004 Jun;99 Suppl 1(s1):51-77. [Medline] [CrossRef]
- Niaura R. Cognitive social learning and related perspectives on drug craving. Addiction 2000 Aug;95 Suppl 2(8):S155-S163. [Medline] [CrossRef]
- Shiffman S, Kassel J, Gwaltney C, McChargue D. Relapse prevention for smoking. In: Marlatt GA, Donovan DM, editors. Relapse Prevention: Maintenance Strategies in the Treatment of Addictive Behaviors. 2nd edition. New York: Guilford Press; 2005:92-129.
- Marlatt GA, Donovan DM. Relapse Prevention: Maintenance Strategies in the Treatment of Addictive Behaviors. 2nd edition. New York: Guilford Press; 2005.
- Mahoney MJ. Self-Change: Strategies for Solving Personal Problems. New York: WW Norton; 1979.
- Rachlin H. The Science of Self-Control. Cambridge, MA: Harvard University Press; 2000.
- King DK, Estabrooks PA, Strycker LA, Toobert DJ, Bull SS, Glasgow RE. Outcomes of a multifaceted physical activity regimen as part of a diabetes self-management intervention. Ann Behav Med 2006 Apr;31(2):128-137. [Medline] [CrossRef]
- Doshi A, Patrick K, Sallis JF, Calfas K. Evaluation of physical activity web sites for use of behavior change theories. Ann Behav Med 2003;25(2):105-111. [Medline] [CrossRef]
- McKay HG, King D, Eakin EG, Seeley JR, Glasgow RE. The diabetes network internet-based physical activity intervention: a randomized pilot study. Diabetes Care 2001 Aug;24(8):1328-1334 [FREE Full text] [Medline] [CrossRef]
- Hughes JR, Keely JP, Niaura RS, Ossip-Klein DJ, Richmond RL, Swan GE. Measures of abstinence in clinical trials: issues and recommendations. Nicotine Tob Res 2003 Feb;5(1):13-25. [Medline]
- Glasgow RE, Mullooly JP, Vogt TM, Stevens VJ, Lichtenstein E, Hollis JF, et al. Biochemical validation of smoking status: pros, cons, and data from four low-intensity intervention trials. Addict Behav 1993;18(5):511-527. [Medline] [CrossRef]
- Severson HH, Andrews JA, Lichtenstein E, Gordon JS, Barckley M, Akers L. A self-help cessation program for smokeless tobacco users: comparison of two interventions. Nicotine Tob Res 2000 Nov;2(4):363-370. [Medline] [CrossRef]
- Severson HH, Andrews JA, Lichtenstein E, Danaher BG, Akers L. Self-help cessation programs for smokeless tobacco users: long-term follow-up of a randomized trial. Nicotine Tob Res 2007 Feb;9(2):281-289. [Medline] [CrossRef]
- Jack LM, Swan GE, Thompson E, Curry SJ, McAfee T, Dacey S, et al. Bupropion SR and smoking cessation in actual practice: methods for recruitment, screening, and exclusion for a field trial in a managed-care setting. Prev Med 2003 May;36(5):585-593. [Medline] [CrossRef]
- Kenford SL, Fiore MC, Jorenby DE, Smith SS, Wetter D, Baker TB. Predicting smoking cessation. Who will quit with and without the nicotine patch. JAMA 1994 Feb 23;271(8):589-594. [Medline] [CrossRef]
- Stein MD, Anderson BJ, Niaura R. Nicotine replacement therapy: patterns of use after a quit attempt among methadone-maintained smokers. J Gen Intern Med 2006 Jul;21(7):753-757 [FREE Full text] [Medline] [CrossRef]
- Swan GE, Jack LM, Curry S, Chorost M, Javitz H, McAfee T, et al. Bupropion SR and counseling for smoking cessation in actual practice: predictors of outcome. Nicotine Tob Res 2003 Dec;5(6):911-921. [Medline] [CrossRef]
- Piper ME, Guoqing M, Bolt DM, McCarthy DE, Smith SS, Fiore MC, et al. The Wisconsin Predicting Relapse In Smoking Measure (WI-PRISM): a new measure of relapse proneness. Annual Meeting of the Society for Research on Nicotine and Tobacco, February 2004, Scottsdale, AZ .
- Danaher BG, Boles SM, Akers L, Gordon JS, Severson HH. Defining participant exposure measures in Web-based health behavior change programs. J Med Internet Res 2006;8(3):e15 [FREE Full text] [Medline] [CrossRef]
- ; Centers for Disease Control and Prevention. Behavioral Risk Factor Surveillance System (BRFSS) Annual Survey Data. 2002 Survey Data. URL: http://www.cdc.gov/brfss/technical_infodata/surveydata/2002.htm [accessed 2008-05-27] [WebCite Cache]
- Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd edition. New York: John Wiley & Sons, Inc; 2000.
- Singer JD, Willett JB. Modeling the days of our lives: using survival analysis when designing and analyzing longitudinal studies of duration and the timing of events. Psychol Bull 1991:268-290. [CrossRef]
- Singer JD, Willett JB. Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. New York: Oxford University Press; 2003.
- Schaefer JL. Analysis of Incomplete Multivariate Data. London, UK: Chapman & Hill; 1997.
- Japuntich SJ, Zehner ME, Smith SS, Jorenby DE, Valdez JA, Fiore MC, et al. Smoking cessation via the internet: a randomized clinical trial of an internet intervention as adjuvant treatment in a smoking cessation intervention. Nicotine Tob Res 2006 Dec;8 Suppl 1(1):S59-S67. [Medline] [CrossRef]
- Street LL, Luoma JB. Control groups in psychosocial intervention research: ethical and methodological issues. Ethics Behav 2002;12(1):1-30. [Medline] [CrossRef]
- Hedeker D, Mermelstein RJ, Demirtas H. Analysis of binary outcomes with missing data: missing = smoking, last observation carried forward, and a little multiple imputation. Addiction 2007 Oct;102(10):1564-1573. [Medline] [CrossRef]
- Reips UD. Standards for Internet-based experimenting. Exp Psychol 2002;49(4):243-256. [Medline]
- Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11 [FREE Full text] [Medline] [CrossRef]
- Sereika SM, Davis CE. Analysis of clinical trials and treatment nonadherence. In: Burke LE, Ockene IS, editors. Compliance in Healthcare and Research. Armonk, NY: Futura (Blackwell Publishing); 2001:263-284.
- Ershoff DH, Quinn VP, Boyd NR, Stern J, Gregory M, Wirtschafter D. The Kaiser Permanente prenatal smoking-cessation trial: when more isn't better, what is enough? Am J Prev Med 1999 Oct;17(3):161-168. [Medline] [CrossRef]
- Calabrese EJ, Baldwin LA. U-shaped dose-responses in biology, toxicology, and public health. Annu Rev Public Health 2001;22:15-33. [Medline] [CrossRef]
- Christensen H, Mackinnon A. The law of attrition revisited. J Med Internet Res 2006;8(3):e20; author reply e21 [FREE Full text] [Medline] [CrossRef]
- Eysenbach G. Tackling publication bias and selective reporting in health informatics research: register your eHealth trials in the International eHealth Studies Registry. J Med Internet Res 2004 Sep 30;6(3):e35 [FREE Full text] [Medline] [CrossRef]
|PAR-Q: Physical Activity Readiness Questionnaire|
|QSN: Quit Smoking Network|
|RCT: randomized controlled trial|
|SHIP: Smokers’ Health Improvement Program|
Edited by C Norman, S McIntosh, P Selby; submitted 14.11.07; peer-reviewed by JF Etter, C Victor; comments to author 06.01.08; revised version received 22.02.08; accepted 06.07.08; published 18.11.08
© H Garth McKay, Brian G Danaher, John R Seeley, Edward Lichtenstein, Jeff M Gau. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 18.11.2008.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.