This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Concurrent with their enrollment in Web-based Randomized Controlled Trials (RCTs), participants can easily choose to use treatment programs that are not assigned in the study. The prevalence of using non-assigned treatments is largely unknown although it is likely to be related to the extent to which non-assigned treatments are: (a) easy to find and use, (b) low in cost, (c) well publicized, and (d) available from trusted sources. The impact of using other programs—both beneficial and detrimental—warrants additional research investigation.
The aim of this report is to explore the extent to which participants enrolled in a Web-based intervention for smoking cessation used treatment methods that were not explicitly assigned (“non-assigned treatment”). In addition to describing the relation between using non-assigned treatments and smoking cessation outcomes, we also explore the broader issue of non-assigned program use by RCT participants in Web-based behavioral interventions, generally.
We describe the use of other programs (as measured by self-report at the 3-month follow-up assessment) by 1028 participants who were randomized to the Web-based SHIP (Smokers’ Health Improvement Program) RCT which compared the Quit Smoking Network (QSN) treatment program and the Active Lives control condition. We examine the extent to which pharmacotherapy products were used by participants in the QSN condition (which explicitly recommended their use) and the Active Lives condition (which purposefully omitted mention of the use of pharmacotherapy). We also test for any between-condition impact of using non-assigned treatments and pharmacotherapy products on smoking cessation outcomes.
A total of 24.1% (248/1028) participants reported using one or more smoking cessation treatment programs that were not explicitly recommended or assigned in their treatment protocol. Types of non-assigned treatments used in this manner included individual counseling (1.7%), group counseling (2.3%), hypnotherapy/acupuncture (4.5%), pamphlets/books (12.6%), and other Web-based smoking cessation programs (9.0%). Participants who used non-assigned treatments were more likely to be female and have at least a high school education. Use of non-assigned Web programs was related to greater levels of self-reported smoking cessation measured at the 3-month assessment (OR = 2.63, CI = 1.67 - 4.14,
A noteworthy proportion of individuals recruited via the Internet to participate in a Web-based intervention used treatment programs and tools not formally assigned as a part of their research protocol. We consider factors likely to influence using non-assigned treatments and suggest ways that future research can begin to study more fully this important phenomenon which is likely to be found in any type of research, but may be particularly pronounced in minimal contact, Web-based intervention trials.
Research interest in Web-based health behavior change interventions is growing rapidly [
In many other instances, researchers have not acknowledged or even reported upon the prevalence and impact of RCT participant use of non-assigned treatments in this manner. Literature germane to this topic includes treatment debriefing (eg, [
The SHIP smoking cessation RCT used online recruitment methods (ie, ad placement on Google and Yahoo search engines and links to affiliated sites) to enroll 2318 smokers from the US and Canada to participate in a randomized controlled trial. The trial was not registered, because enrollment started in spring 2005, before trial registration became mandatory. Prospective participants visited the recruitment website where they completed an online screening survey that included the 8-item Physical Activity Readiness Questionnaire (PAR-Q) [
Smokers who completed the screening and consent stages were randomized using a computer-based vector method to one of two Web-based programs: (a) the Quit Smoking Network (QSN) condition (N = 1159) or (b) the Active Lives control condition (N = 1159). Baseline data of 2318 study participants showed that most were women (70.5%), White (86.6%), urban (80.3%), married (61.6%), had at least some college education (68.2%), and smoked 1 - 2 packs of cigarettes each day (78.5%).
When study participants first used the Web-based QSN program, they were required to move through a series of Web pages that introduced key concepts and strategies of a combined behavioral-pharmacologic program for quitting smoking. Thereafter—and during subsequent visits—participants were free to choose any of a broad array of additional content on quitting and maintaining nonsmoking. The behavioral intervention was based on Social Cognitive Theory [
The QSN program strongly advocated the use of pharmacological adjuncts and it contained a number of Web pages devoted to the use of Nicotine Replacement Therapy (NRT) and Zyban®. These Web pages provided an explanation of how to use these products, photos of representative products, supportive videos of smokers, interactive questions designed to elicit participant commitment to use these products, and agreement to see a doctor in order to obtain a prescription. NRT products included nicotine gum, patch, lozenge, spray, and inhaler.
The Web-based Active Lives control condition was a content-rich, multiple-module Web-based program that encouraged smokers to develop a personal physical activity program in order to become more fit which, in turn, would help them to quit smoking. The program guided each participant through a multi-step plan that included a motivational component (exploration of the benefits of physical activity and a clarification of personal goals and barriers), a behavioral action plan with extensive tracking features (eg, weekly activity schedules personalized to each participant’s schedule and types of activities), additional online resources (articles and “tip” sheets), and access to a Web Forum for peer support (distinct from the aforementioned peer forum in the QSN program). In contrast to the QSN condition, the Active Lives control condition purposefully omitted any reference to the use of pharmacotherapy (NRT or Zyban®).
Both the QSN and Active Lives programs encouraged participants to use the smoking cessation approaches featured in each website. However, participants were not explicitly cautioned against using other treatment programs or resources during and/or following their involvement with this study.
Assessment data were collected at screening, baseline, and at 3- and 6-month follow-up assessments. Assessments were completed either online or via phone.
Non-assigned treatment use was measured by two items on the 3-month follow-up assessment. The first item asked: Which of the following products or methods have you tried in the last 3 months? (check all that apply). Answer options included treatment methods assigned in the QSN intervention condition but not in the Active Lives control condition (nicotine gum, nicotine patch, nicotine lozenge, nicotine spray, nicotine inhaler, other nicotine replacement product, Zyban), treatment methods that were not assigned in either the treatment or control condition (group cessation program or class, individual counseling [including by telephone], hypnosis or acupuncture, pamphlets or books), or none of the above. A separate item asked: Have you used any other Internet smoking cessation programs since first using the QSN/Active Lives program?
We created two composite measures of non-assigned treatment usage: one measure was defined as the sum of non-assigned treatments reportedly used (score ranged from 0 - 5; treatments included individual counseling, group counseling, hypnotherapy/acupuncture, other Web programs, and pamphlets/books), and the other composite was defined as the yes/no dichotomy describing whether any of these non-assigned treatments had been used.
As noted above, participants were asked (yes/no) whether they had used any pharmacotherapy products (nicotine gum, patch, lozenge, spray, and inhaler) or Zyban® since the start of their involvement in the SHIP study. Use of NRT products was explained and strongly recommended in the QSN condition, but the topic was purposefully omitted in the Active Lives control condition. We created two composite measures of using pharmacotherapy: one measure was defined as the sum of non-assigned treatments reportedly used (nicotine gum, patch, lozenge, spray, inhaler, and Zyban®), and the score ranged from 0 - 6; and the other measure was defined as the yes/no dichotomy describing whether any of these pharmacotherapy programs had been used.
The extent to which participants accessed their assigned Web-based program was measured unobtrusively using a combination of database tracking and Web-server log analysis [
Participant 7-day point prevalence smoking abstinence was assessed both at 3 and 6 months by asking: Have you smoked any cigarettes in the last week, even a puff? The more rigorous repeated point prevalence of self-reported smoking cessation at both the 3- and 6-month assessments was also used. As with other Web-based programs and large-scale self-help interventions for tobacco cessation (eg, [
We also measured putative predictors of smoking cessation. Baseline assessment included an item about friends’ smoking (Most of my friends and acquaintances smoke [1 = Not true of me at all, 7 = Extremely true of me]), two items on nicotine dependence (I usually want to smoke right after I wake up [1 = Not true of me at all, 7 = Extremely true of me]; How strong are your urges when you first wake up in the morning? [1 = Not strong at all, 7 = Extremely strong]), and five self-efficacy items. The self-efficacy items all used the same 7-point rating scale (1 = Not at all confident, 7 = Very confident), and they included a global item (If you decided to quit smoking, how confident are you that you could quit) and four items that asked about specific settings/circumstances (How confident are you that you can resist smoking when you are feeling bored or restless?; How confident are you that you can resist smoking when you are angry, frustrated, or tense?; How confident are you that you can resist smoking when you drink alcohol?; How confident are you that you can resist smoking when you are around others who are using it?).
Logistic and standard regression tests were used to test the relation between participant characteristics and reported use of non-assigned treatments. Similar analyses were used to test the relation of non-assigned treatment use, controlling for treatment condition, on point prevalence smoking cessation at 3 months, at 6 months, and for repeated point prevalence that considered smoking status at both 3- and 6-month follow-up assessments.
Consistent with many Web-based tobacco cessation interventions, the SHIP trial experienced significant attrition over the follow-up interval. Of the 2318 participants initially randomized, 44.3% (N = 1028) completed the 3-month assessment, 32.8% (N = 909) completed the 6-month assessment, and 27.2% (N = 631) completed both assessments. No between-group differences in attrition were found.
CONSORT diagram for SHIP RCT
A total of 24.1% (248/1028) of participants reported that they had used some other smoking cessation program during the first 3 months they were enrolled in the SHIP trial. The types of non-assigned treatments used depicted in
Participant use of non-assigned treatments: reported at the 3-month follow-up
QSN Intervention | Active Lives Control | Total | |
Non-assigned treatment | N = 524 | N = 504 | N = 1028 |
Individual counseling | 7 (1.3%) | 10 (2.0%) | 17 (1.7%) |
Group counseling | 15 (2.9%) | 9 (1.8%) | 24 (2.3%) |
Hypnotherapy/acupuncture | 22 (4.2%) | 24 (4.8%) | 46 (4.5%) |
Other Web-based programs | 43 (8.2%) | 50 (9.9%) | 93 (9.0%) |
Pamphlets/books | 65 (12.4%) | 65 (12.9%) | 130 (12.6%) |
The QSN intervention condition explained and recommended the use of pharmacotherapy products whereas the Active Lives control condition did not. As can be seen in
Participant use of pharmacotherapy products: reported at the 3-month follow-up
QSN Intervention | Active Lives Control | Total | |
N = 524 | N = 504 | N = 1028 | |
Nicotine gum | 65 (12.4%) | 65 (12.9%) | 130 (12.6%) |
Nicotine patch | 143 (27.3%) | 124 (24.6%) | 267 (26.0%) |
Nicotine lozenge | 40 (7.6%) | 25 (5.0%) | 65 (6.3%) |
Nicotine spray | 5 (1.0%) | 5 (1.0%) | 10 (1.0%) |
Nicotine inhaler | 22 (4.2%) | 14 (2.8%) | 36 (3.5%) |
Zyban® | 71 (13.5%) | 56 (11.1%) | 127 (12.4%) |
Each of six participant baseline characteristics (age, gender, marital status, education, rurality, cigarettes smoked/day) was tested using univariate logistic regression for its relation to any non-assigned treatment use. Non-assigned treatment use (composite dichotomous yes/no measure) was found to be positively related to being female (OR = 1.90, 95% CI = 1.34 - 2.69,
A Pearson correlation was used to test the relation between participant exposure and the number of non-assigned treatment types used. The result indicated little relation between participant exposure to the assigned Web-based program and the use of non-assigned treatments:
A total of 202 participants reported not smoking at 3 months: 19.6% complete case (202/1028) and 8.7% ITT (202/2318). At the 6-month assessment, 232 participants reported not smoking: 25.5% complete case (232/909) and 10.0% ITT (232/2318). A total of 89 participants who completed both the 3- and 6-month follow-up assessments indicated that they were not smoking on each occasion: 14.1% complete case (89/631) and 3.8% ITT (89/2318). No statistically significant between-group differences in smoking cessation were found at these assessment points [
We used univariate logistic regression to determine the relation of each of the five types of non-assigned treatment use and smoking cessation at 3 months, at 6 months, and the 3- and 6-month repeated point prevalence measure. Only use of other Web programs was found to be related to smoking cessation: it was positively related at the 3-month assessment (OR = 2.63, CI = 1.67 - 4.14,
In addition, a test for the moderator effect of condition and non-assigned treatment usage on smoking cessation failed to find any noteworthy interaction effects at either the 3- or the 6-month outcome. Indeed, when we eliminated from the analysis data of participants who indicated that they had used non-assigned treatments, no effect for condition on smoking cessation outcome emerged at 3 months, 6 months, or the combined 3- and 6-month assessments.
Univariate logistic regression revealed four putative predictors of smoking cessation to be significantly related to non-assigned treatment use: self-efficacy to quit when using alcohol (OR = 1.09, CI = 1.00 - 1.18,
Univariate logistic regression revealed that the sum of pharmacotherapy products reported at 3 months used was unrelated to smoking cessation at 3 months (OR = 1.07, CI = .89 - 1.27,
Strengths of the current research include the successful use of online marketing strategies to recruit a large sample of 2318 participants and our use of a RCT methodology. Limitations include noteworthy participant attrition—an outcome that has been reported in other Web-based tobacco cessation studies [
Additional debriefing questions were not included in the assessment that could have helped to illuminate reasons for using non-assigned treatments. For example, questions could have probed participants’ attitudes about, and reasons for, using other smoking cessation programs, and the extent that they thought non-assigned treatments were helpful and personally relevant. It would be interesting to know whether study participants felt that outside programs were relatively more or less helpful than the treatment methods that were assigned. In addition, we could have asked more specifically about the timing of when participants used non-assigned treatments.
The incidence of using non-assigned treatments is quite difficult to gauge given that most publications fail to report upon this phenomenon. An exception is Strecher and colleagues [
The phenomenon of using non-assigned treatments may be particularly likely among participants of Web-based RCTs who demonstrated their Web foraging skills [
The frequency and timing of asking participants about their use of non-assigned treatments deserves careful consideration. Because of the substantial attrition found in many Web-based intervention trials [
It is impractical to require Web-based RCT participants to refrain from using alternative treatment programs or to avoid treatment-seeking from other sources. We recommend that the use of non-assigned treatments should not be grounds for participant exclusion from Web-based behavior change interventions. Instead, Web-based interventions should be evaluated as being part of a larger fabric of ongoing self-help and personal improvement programs that people engage in to accomplish important personal behavioral changes. Before they become study participants—and possibly during the time that they are study participants—individuals are likely to be seeking out available resources, including those readily available on the Internet, some of which they may use in making a serious attempt to change their behavior, as in trying to quit smoking [
Research may show that it is beneficial to encourage participants to use other treatment resources to complement what they learn about in the behavior change program presented in their RCT. However, engaging in multiple concurrent treatments—some of which might be contradictory—could be counterproductive [
Thanks are extended to John Noell (the initial Principal Investigator in the SHIP trial) and Edward Feil both of whom played a key role in the early development phases of the SHIP project. In addition, we acknowledge the contribution of Ron Williams, who had primary responsibilities for programming and website development, and Jeff M. Gau, who made major contributions to the data analysis. This research was supported by grant R01-CA79946 from the National Cancer Institute.
None declared.
intent-to-treat
nicotine replacement therapy
physical activity readiness questionnaire
quit smoking network
randomized controlled trial
smokers’ health improvement program