The importance of smoking on public health is undeniable: “Cigarette smoking remains the leading cause of preventable mortality in the United States, resulting in nearly 16 million deaths since the first Surgeon General’s report on smoking and health in 1964” [1]. In addition to the profound deleterious impact on smokers’ health, “secondhand smoke causes premature death and disease in children and in adults who do not smoke” [2,3]. In response to the public health risks associated with smoking, numerous research projects have examined tobacco cessation approaches that are based in clinical settings as well as public health methods that permit wider dissemination [4-6].

One nascent area of development for public health smoking cessation programs involves the use of Internet-based interventions. To date, only a handful of published studies have described the efficacy/effectiveness of Web-based tobacco cessation programs. Several reports have described promising results of trials with participants from commercial or tobacco control agency Web-based programs for smokers [7-10]. Other published studies have described initial feasibility studies of Web-based tobacco cessation programs [11,12]. A few randomized controlled trials (RCTs) have been conducted [9,13-19], and their results have been generally encouraging [20]. However, interpretation of the randomized trials is complicated by short follow-up intervals, for example, 6 weeks in Strecher et al [16], 2.5 months in Etter [8], and 3 months in Lenert et al [15], Pike et al [9], and Swartz et al [18]. Additional trials with longer term follow-up assessments examining the use of Web-based interventions for smoking control are clearly warranted.

The current paper describes results of the Smokers’ Health Improvement Program (SHIP) RCT that examined the 3- and 6-month outcomes for two Web-based programs: (1) an intervention condition (Quit Smoking Network; QSN) that provided users with extensive information and behavioral strategies drawn from clinic-based and self-help smoking cessation programs, or (2) a Web-based control condition that focused on increasing physical activity (Active Lives). Currently, because grant funding has ended, neither the QSN nor Active Lives website is currently available for review. However, screenshots of the programs are included in the Multimedia Appendix.

In addition to primary outcome measures of tobacco abstinence, we sought to examine the impact of condition on secondary outcomes, including participant exposure to program content, physical activity, and pharmacotherapy use. We also sought to test the putative predictors of outcome. Finally, we assessed the extent to which participants found their assigned program easy to use.

The outcome results did not support our hypothesis that the QSN online smoking cessation intervention would be more effective than a credible control condition. The unremarkable impact of the QSN condition relative to the Active Lives condition is particularly surprising given that the Active Lives control condition presented very few strategies for quitting smoking since it was largely focused on helping participants improve their personal level of physical activity. The absolute level of nonsmoking at 6 months—less than 4% abstinence using intent-to-treat analysis—is less than results for other Web-based smoking cessation programs reported by Muñoz et al (5.6% to 26.0% abstinence at 6 months) [13]. However, with respect to the nonsignificant finding between conditions, the engagement of physical activity among the Active Lives participants was noteworthy. These data reflect their relatively high level of adherence to the recommended behavior change goal. Almost 80% of participants reported that they were engaged in moderate physical activity at the 6-month follow-up assessment. This implies that our control group was actively following the recommendations suggesting the importance of engaging in physical activity in aiding quit attempts and might explain, in part, the relative lack of difference in findings between the groups on quit success.

There are noteworthy strengths and limitations to consider when interpreting these findings. Strengths include the large sample of 2318 participants, the fact that the Web-based programs were fully automated to assure high fidelity of content delivery, and that the study used a RCT methodology.

One important limitation involved participant attrition (failure to complete follow-up assessments). At the 3-month follow-up, we experienced a 55.6% (1289/2318) attrition rate, which is larger than the 44% attrition rate reported by Swartz et al [18] but comparable to the levels reported for other Web-based smoking cessation programs: 57.2% by Stoddard et al [11], 62.5% by Strecher et al [16], 70.7% by Cobb et al [7], and 64.6% by Etter [8]. At the 6-month follow-up, we experienced a 60.8% (1409/2318) attrition rate, which is roughly comparable to the attrition at 6 months reported by Muñoz et al [13] for four studies of English- and Spanish-language Web-based smoking cessation programs: 73.9% (2051/2774), 69.9% (491/702), 65.7% (184/280), and 35.4% (102/288).

Although mean program exposure measures (especially duration) favored the QSN program when compared with the Active Lives control condition, the extent of these observed differences was not as large as had been expected. Nor did we find a dose-response relationship between program exposure and smoking abstinence at follow-up. Using median data, participants in the current study used the program for a single visit that lasted about 10 minutes.

Differences in study design among published studies of Web-based smoking cessation interventions and the nascent stage of the science make it difficult to generalize our program exposure results to the available body of research in this area. For example, some researchers have not reported exposure data [13,15,18], while others [8] reported number of pages viewed and average visit duration but not a precise calculation of exposure. The pattern of program visits (mean = 2.14, median = 1.0) observed for both conditions in this study is lower than the findings reported by Swartz et al [18] and Lenert et al [14] but roughly consistent with results of other online cessation programs reported by Muñoz et al [13]. In their comparison of five different Web-based smoking cessation interventions, Pike et al [9] noted that the two websites with the highest “utilization rates” had 9.7 and 6.0 visits per participant, while the visit rate on the remaining websites ranged from 1.8 to 2.2 visits.

It is important to note that Japuntich et al [51] reported much higher levels of exposure to an adjunctive Web-based intervention by participants who had multiple in-person contacts with research project staff (including a physical exam, several counseling visits, and five in-person follow-up assessments). It is reasonable to assume that there are likely to be significant differences between adjunctive use of Web-based interventions compared to fully automatic Web-based interventions like ours that do not involve any in-person contact and that deliver the intervention only by a rules-driven algorithm.

Nonetheless, there remains the opportunity to improve participant engagement in online smoking cessation programs. For example, future research should consider testing whether engagement might be facilitated by changing program content from visit to visit (enhancing vitality), by using more effective tailoring to improve the relevance of program content, and/or by using innovative reminders (eg, some combination of email, regular mail, text messages, e-cards) that encourage multiple program visits.

Analyses of pharmacotherapy usage were supportive of the fact that the QSN condition compared to the control condition encouraged significantly greater use of this treatment approach (more than 60% vs approximately 47%), although it is important to acknowledge that almost half of participants in the control condition used pharmacotherapy without being explicitly told to do so. Similarly, Swartz et al [18] reported that a majority of wait-list control participants indicated that they used pharmacotherapy products.

One interesting possibility is that our use of a physical activity control condition may have inadvertently jeopardized, to some extent, the generalizability of our results. Specifically, because of concerns about health among participants in the Active Lives condition, we excluded 61 individuals (prior to randomization) who had positive answers on the PAR-Q assessment, and an additional 48 individuals declined to participate because they did not want to increase their level of exercise activity. These 109 individuals would not have been excluded from a typical smoking cessation RCT. However, it remains for further research to determine whether these exclusion criteria may have had the effect of excluding individuals who otherwise would have been successful quitters using the treatment condition.

We considered possible control/comparison conditions for the present study. A no-treatment or waiting list control group is often recommended in order to reduce the likelihood of a type II (false negative) error when an intervention is expected to produce small effects. But we concluded that offering participants no treatment or delayed treatment would have been unhelpful because waiting list controls provide only limited information [52]. Moreover, from a pragmatic perspective, we were concerned that individuals assigned to a waiting list condition would have little reason to remain involved and would therefore be more likely to be lost to follow-up. When using an intent-to-treat model that defines missing = smoking, such differential attrition would have biased results in favor of the intervention condition or type I error [53]. Thus, we used a control/comparison condition that offered a face-valid, credible, Web-based intervention for smoking cessation that did not contain what have been shown to be the active ingredients of evidence-based smoking cessation intervention. It is reasonable to question whether a better choice of control condition would have been a basic or static information website similar to what has been used in other RCTs of tobacco cessation interventions (eg, [16,19]).

Our remarkably successful use of online recruitment may have resulted in recruiting smokers who were less likely (1) to remain fully involved over time in a research program and (2) to quit smoking. It was quite easy for prospective participants to enroll—we estimate that it would require less than 15 minutes from clicking on a link in a Google ad listing to completing the screening, online consent, registration, and baseline assessment. The absence of measures describing ease of enrollment (either self-report or using a measure of convenience like elapsed time) makes it impossible for us to discuss further the extent to which the present study may have had easier enrollment than other Web-based behavioral interventions. Adding more barriers/hurdles to this process would very possibly have increased engagement and reduced program attrition because only motivated individuals would have been able to participate [54,55]. But, while culling out less motivated individuals, this approach might spuriously inflate absolute effect sizes while reducing external generalizability [56]. It remains for future research to determine the extent to which open online enrollment with few barriers for entry results in greater attrition.

Another likely limitation of the QSN intervention is that it may have been too expansive; that is, it may have offered too much (sometimes redundant) content that forced users to navigate through a complex information architecture, which reduced utility and encouraged attrition. We obtained ratings of program usability at follow-up, but, of course, interpretation of such data is constrained by the fact that they are drawn from a minority of original participants who completed follow-up assessments. Content duplication and/or complex intervention design can erode therapeutic impact—more is not necessarily better [57]. Indeed, the likely mechanisms of change may not be best described by a linear dose-response relationship [58,59].

In many ways, the results of the current research underscore the complexity of developing and then evaluating Web-based interventions for smoking cessation. We used an intervention and an attention placebo control that recommended physical activity as a smoking cessation tool. Our initial plan was to recruit through worksites as had been done with success in other Web-based smoking cessation research [18]. Because we found that worksite recruitment produced little in the way of results, we turned to online recruitment tied to popular Web search engines, particularly Google. Using this approach, we were able to rapidly randomize over 2300 participants to our RCT of two Web-based smoking cessation programs that had been carefully crafted to provide users with the content they needed in an online context that they would find interesting and engaging.

In summary, the results of this Web-based smoking cessation intervention RCT failed to confirm our hypotheses. Negative findings play an essential role in the development of science [60] and are particularly illuminating with regard to shaping a reasoned appreciation for the complexity of creating, delivering, and evaluating online health behavior change interventions. As we perform empirical tests of evolving interventions in this nascent field, we need to learn from both negative and positive outcomes as we strive to understand the factors that are associated with participant recruitment, program exposure, content tailoring, and participant attrition.