This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
The Internet has great potential to provide assistance to millions of smokers who seek help with quitting smoking.
The goals of this study were to assess the content and the quality of smoking cessation treatments most likely to be encountered by smokers seeking treatment on the Internet and to examine differences in quality between current websites and those reviewed in 2004.
Internet searches for smoking cessation were designed to mimic the search patterns of most Internet users. PhD-level specialists in tobacco cessation treatments used standardized procedures to review the content of each website, assess the degree to which each site covered key components of evidence-based treatment as described in US national guidelines, determine the accuracy of information presented, and evaluate the use of website interactivity. Results of the current study were compared to results obtained in a prior review.
Most websites retrieved in the search met exclusion criteria and were not included in the final analyses in both the current (74%, 65/88) and the prior study (77%, 156/202). In both studies, the majority of websites were excluded because they sold cessation-related products but did not provide treatment recommended by the Public Health Service guidelines. Of the 23 websites included in the current study, 26% (n = 6) provided only minimal coverage (brief mention) of key components of tobacco treatment. However, compared to the earlier study, websites included in the present study scored significantly higher in quality ratings in four areas: providing advice to quit (
While the quality of treatment offered in WATIs has improved since our previous review in 2004, there is substantial room for further improvement to ensure that smokers are offered high-quality, evidence-based treatments. It is not clear what degree of informational detail and interactivity is optimal for Web-based smoking cessation treatments. Additional research is needed to understand how to maximize the interactive capabilities of the Internet to produce and sustain population-based health behavior change.
The most recent national data show that 20.9% of US adults are current smokers [
The Internet is a widely accessible delivery channel that has great potential to reach millions of smokers with evidence-based treatments. Increasingly, smokers are using the Internet for cessation assistance; recent reports estimate that, annually, over 10 million Americans search the Internet for information and support to quit smoking [
In response to this demand, there are now hundreds of smoking cessation websites. Several earlier reviews have found that most sites were of mediocre quality [
Based on these developments, we were interested in determining whether the landscape of smoking cessation websites had changed since our previous review. Our goals in this study were (1) to assess the content and the quality of Web-assisted tobacco interventions (WATIs) most likely to be encountered by smokers and (2) to determine the extent to which WATIs have changed since our earlier review [
The first step was to locate WATI websites. Our approach was designed to mimic the search patterns of most Internet users. Several reports have shown that the majority of Internet users tend to use only one search engine and stop at the first page of search returns [
Standardized Internet searches were conducted by entering “smoking,” “smoking cessation,” “quit smoking,” and “stop smoking” as the Boolean text string into each of the selected search engines. We compiled a list of all websites retrieved within the first page of search results for each search engine into a list of potential WATI sites for review. Redundant sites returned by more than one search were eliminated from the list, resulting in a final list of 88 unique websites. Using a standardized form, trained coders reviewed the content of each website to determine whether it should be included in this review. Coders reviewed the website’s home page and first level and second level content pages. If relevant content was not detected within these three levels of exploration, the site was excluded from our review. Websites that did not provide direct tobacco treatment services via the Internet were excluded from analysis. For the purposes of this study, “treatment” was operationally defined as the provision of organized, directive information and support services relevant to the process of quitting smoking. Websites were excluded from analysis if they met one or more of the following criteria: (1) product sales only (no treatment components as recommended by the Public Health Service [PHS] guidelines were available on the website itself), (2) libraries (sites that contained articles about smoking, smoking cessation, tobacco policy, advocacy, addiction, or other related topics, but which provided no clear organization or guidance for the smoker who wished to quit), (3) links (sites that only contained links to other sites, including website links and references to hotline phone numbers and bookstores [eg, Amazon.com]), (4) clinics and practitioners advertising face-to-face services, (5) advocacy and political action sites, (6) professional education and information sites designed for health care providers, (7) dead or abandoned websites (eg, a return of “404 file not found” or similar), and/or (8) site content was not smoking-related. The 23 websites reviewed in the current study and the websites reviewed in the 2004 study are presented in
Websites reviewed
2004 | 2007 |
1. QuitNet.coma | 1. QuitNet.coma |
2. SurgeonGeneral.gov/tobacco | 2. SurgeonGeneral.gov/tobacco |
3. Quitsmoking.about.com | 3. Quitsmoking.about.coma |
4. TryToStop.org | 4. TryToStop.org |
5. Cancer.org/tobacco | 5. Cancer.org/tobacco |
6. LungUSA.orga | 6. LungUSA.org |
7. CDC.gov/tobacco | 7. CDC.gov/tobacco |
8. http://equinox.unr.edu/ homepage/shubinsk | 8. http://equinox.unr.edu/ homepage/shubinsk |
9. Quit.org.au | 9. Quit.org.au |
10. stop-tabac.cha | 10. Smoking-cessation.orga |
11. Cancer.ca/tobaccoa | 11. GivingUpSmoking.co.uka |
12. ashline.orga | 12. StopSmokingCenter.neta |
13. QuitSmokingSupport.com | 13. WebMD.com |
14. MindFocus.com | 14. N/Ab |
15. QuitSmokingUK.com | 15. Best-StopSmokingProducts.org |
16. Nicotine-Anonymous.org | 16. Quit-smoking-guide.com |
17. QuitSmokingIn7Days.com | 17. RealOvercoming.com |
18. Habitrol.com | 18. WhyQuit.com |
19. DrKoop.com | 19. Quit-smoking-review.com |
20. QuitSmoking.com | 20. SmokeFree.gov |
21. nicorette.com | 21. Quit.com |
22. Zyban.com | 22. MostImportantGift.com |
23. LifeSign.com | 23. SmokingTown.com |
24. Smokehelp.org | |
25. TobaccoFree.com | |
26. HeliosHealth.com/quit_smoking | |
27. 123-quit-smoking.com | |
28. Smokestoppers.com | |
29. LifeClinic.com/focus/smoking | |
30. Quit4life.com/html | |
31. QuitSmokingHelper.com | |
32. HeartScreen.com/smoking_info.html | |
33. SmokingHealthLine.com | |
34. QuitCommit.com | |
35. QuitSmokingOnLine.com | |
36. QuitTobacco.org | |
37. UCanQuit.com | |
38. QuitTobacco.com | |
39. WellMD.com/QuitSmokingMD.htm | |
40. AHCPR.gov /consumer/helpsmok.htm | |
41. MedUMich.edu/1libr/primry/life04.htm | |
42. Worldzone.net/health/quitsmoking | |
43. InfoTobacco.com | |
44. Hoptechno.com/book43.htm | |
45. SmokeFreeVirginia.org | |
46. MiddlesexHealth.org/health/smoking |
aFive highest ranked websites for coverage of key topic areas.
bNo longer a smoking cessation website as of March 2008.
The first assessment instrument used in this study was the Smoking Treatment Scale - Content (STS-C). Details of the development of the STS-C are described elsewhere [
The second assessment instrument used was the Smoking Treatment Scale - Rating (STS-R), which was developed to provide numeric ratings of quality of coverage for each of the key components of treatment documented in the STS-C. Development of the STS-R is described in detail elsewhere [
Accuracy was rated on a 3-point scale: 3 = “totally correct,” 2 = “mostly correct,” and 1 = “significant misinformation or potentially dangerous errors.” When no inaccurate information was observed, the website received a 3 for that specific component. Where inaccurate information was detected, a rating of 2 was given if the rater judged that the discrepancy was minor and would be unlikely to have harmful effects on site users. In cases where inaccurate information could be potentially dangerous to users (eg, suggesting only palliative remedies for symptoms that could be indicative of nicotine toxicity), a rating of 1 was given.
Reviewers also rated (“yes/no”) whether the website incorporated a user-interactive feature for key treatment components. Interactive features include any content-related user input that results in feedback from the website. Examples of interactive features include entering a target quit date that subsequently generates a quitting calendar or follow-up contact via email; quizzes and assessments that generate individually tailored feedback; chat rooms, bulletin boards, or other interactive community features; interactive recommendations for pharmacotherapy; or the availability of an online pharmacy where medicine could be purchased.
All reviewers were PhD specialists in smoking cessation research and treatment. Reviewers were selected for their clinical or scientific experience, familiarity with the PHS guidelines for the treatment of tobacco dependence, and current research interests or clinical specialization in tobacco dependence treatment. No reviewer had consulted for or had any financial interest or involvement with any of the websites they were assigned to review. Four additional websites (not included in the 23 sites in the analyses) were reviewed for training purposes. After each training review, panel members met to discuss the review process, compare outcomes, and resolve discrepancies. Reviews of each site were conducted independently by two reviewers assigned to each website. Each reviewer used the standardized assessment instruments, which were provided with detailed instructions. Websites were first assessed for content using the STS-C. Results of the content review were used to assign numerical ratings of content quality using the STS-R.
All data were analyzed using SPSS 13.0 statistical software for the PC (SPSS Inc, Chicago, IL, USA). The unit of analysis was the specific URL for all assessments. Interrater reliability was computed for all items on the STS-R. A standard measure of reliability was calculated, computed as the correlation in ratings between reviewers assigned to the same website. Two reviews were included in each calculation of interrater reliability. The overall interrater reliability of the STS-R kappa was .76 or greater for all items, ranging from .76 to .89. Frequency distributions were calculated for each item on both assessment instruments. To examine changes in the quality of WATIs over time, data from the current analyses were compared to the database used in our prior study [
Of the original 88 websites returned from the searches, 65 (74%) were excluded from the review. The most common reasons for exclusion were product sales without smoking cessation treatment available directly on the website (66.2% of excluded sites, 43/65), unguided library of articles (21.5%, 14/65), and websites that only provided links to other websites (16.9%, 11/65). Twenty-three percent of websites met more than one exclusion criteria.
Using the PHS clinical practice guideline [
While a large majority (87%, 20/23) of websites recommended the use of pharmacotherapy for smoking cessation, two sites warned users against using nicotine replacement therapy (NRT), and one site suggested that while NRT was useful, herbal preparations were preferable (ie, “equally effective with fewer side effects”). Among sites recommending medications, only 55% (11/20) provided explanations of how these medications worked, 45% (9/20) gave instructions on how to use these products, and only 35% (7/20) assessed nicotine dependence. Approximately, one-quarter of websites (6/23) asked users to identify negative consequences of smoking and benefits of quitting that were personally relevant.
Of the 23 websites reviewed that provided smoking cessation treatment, 26% (6/23) did not score above 2 (minimal) for any of the key treatment components. Areas most likely to be covered included “provide practical counseling” (100%), “assist with a quit plan” (91%, 21/23), and “recommend pharmacotherapy” (87%, 20/23). Only 47% of sites (11/23) provided more than adequate or extensive coverage (score of 4 or 5) for any key component. The key components most likely to be given extensive coverage were “assist with a quit plan” (22%, 5/23), “provide practical counseling” (26%, 6/23), and “enhance motivation” (relevance = 22%, 5/23; risks = 26%, 6/23). In contrast, providing clear, strong, and personalized advice to quit (0%) and arranging follow-up contact (4%, 1/23) were least likely to be treated extensively across websites. These results are presented in
Overall, the accuracy of information provided by most websites was generally high. Reviewers noted no inaccurate information in 69% of websites (16/23). Minor errors were noted in about 30% of websites (7/23) and were most likely to be found for these key components: “assist with a quit plan” (22%, 5/23), “provide practical counseling” (30%, 7/23), and “recommend pharmacotherapy” (35%, 8/23). For example, in the area of “assist with a quit plan,” one site recommended against setting a target quit day or planning ahead. In “provide practical counseling,” some sites provided links to unproven treatments or offered advice that minimized the risk of drinking alcohol while quitting. Inaccurate information regarding pharmacotherapy included recommending hypnosis as “proven to be more than three times more effective than nicotine replacement,” recommending unproven (typically “herbal” or “laser”) remedies, and advising against using nicotine replacement (eg, referring to NRT as a “natural poison” while endorsing herbal remedies). More than 17% of websites (4/23) contained serious or potentially dangerous errors with regard to pharmacotherapy guidance. For example, one website recommended the use of relaxation techniques to reduce symptoms that could indicate nicotine toxicity; this recommendation made no mention of modifying the dosage of NRT or consulting a physician. In assisting with a quit plan, one website advised against making any specific plans to quit and advised that “cold turkey” was the only way to quit.
We examined data across all key treatment components, regardless of whether any coverage was provided for that key component, and found that 56% of websites (13/23) provided at least one interactive feature, and 39% (9/23) provided two or more interactive features. Among websites that provided coverage for the relevant key treatment content area, the topics most likely to have interactive features were in the areas of “provide social support” (78%, 15/19), “recommend pharmacotherapy” (45%, 9/20), and “enhance motivation - risks” (47%, 8/17). Only one-third of websites (8/23) used interactive features to assess readiness to quit smoking. The remaining sites asked users to select content based on perceived readiness to quit smoking. Approximately one-third (35%, 8/23) of websites contained links to online pharmacies.
Website coverage, accuracy, and interactivity for key components of tobacco dependence treatment; percentages of all sites reviewed (n = 23) within each category
Coverage |
Accuracy |
Interactive (Is feature Interactive?) | |||||||
None | Minimal | Adequate | More Than Adequate | Extensive | Incorrect or Potentially Dangerous | Mostly Correct/Small Errors | Totally Correct/No Errors | Yes | |
1. Advise every tobacco user to quit: strong | 61 | 17 | 13 | 9 | 0 | 0 | 0 | 100 | 0 |
2. Advise every tobacco user to quit: personalized | 65 | 9 | 9 | 17 | 0 | 0 | 5 | 95 | 14 |
3. Assess readiness to quit | 65 | 13 | 9 | 13 | 0 | 0 | 5 | 95 | 15 |
4. Assist with quit plan | 9 | 35 | 17 | 17 | 22 | 4 | 17 | 78 | 13 |
5. Provide practical counseling | 0 | 30 | 30 | 13 | 26 | 0 | 30 | 70 | 22 |
6. Provide intra-treatment social support | 18 | 27 | 27 | 9 | 18 | 0 | 0 | 100 | 40 |
7. Recommend pharmacotherapy | 13 | 44 | 22 | 17 | 4 | 17 | 17 | 65 | 30 |
8. Arrange follow-up | 78 | 4 | 4 | 4 | 9 | 0 | 0 | 100 | 19 |
|
|||||||||
9. Relevance | 22 | 26 | 13 | 17 | 22 | 0 | 0 | 100 | 17 |
10. Risks | 26 | 22 | 9 | 17 | 26 | 0 | 5 | 95 | 27 |
11. Rewards | 26 | 13 | 17 | 30 | 13 | 0 | 5 | 95 | 9 |
12. Roadblocks | 13 | 30 | 22 | 22 | 13 | 0 | 0 | 100 | 9 |
Differences in mean coverage rating scores of websites between 2004 and 2007 reviewa
2004 |
2007 |
Difference in Score |
|
|
1. Advise every tobacco user to quit: strong | 1.41 (.68) | 1.70 (1.0) | 0.29 | .15 |
2. Advise every tobacco user to quit: personalized | 1.33 (.59) | 1.78 (1.2) | 0.45 | .05 |
3. Assess readiness to quit | 1.63 (.77) | 1.70 (1.1) | 0.07 | .48 |
4. Assist with quit plan | 2.74 (.65) | 3.09 (1.3) | 0.35 | .24 |
5. Provide practical counseling | 2.74 (.58) | 3.30 (1.1) | 0.56 | .02 |
6. Provide intra-treatment social support | 2.65 (.58) | 2.74 (1.3) | 0.09 | .49 |
7. Recommend pharmacotherapy | 2.41 (.70) | 2.57 (1.0) | 0.16 | .35 |
8. Arrange follow-up | 1.30 (.71) | 1.61 (1.2) | 0.31 | .74 |
|
||||
9. Relevance | 2.30 (.73) | 2.91 (1.5) | 0.61 | .05 |
10. Risks | 2.13 (.71) | 2.96 (1.6) | 0.83 | < .001 |
11. Rewards | 2.41 (.68) | 2.91 (1.4) | 0.50 | .11 |
12. Roadblocks | 2.57 (.69) | 2.91 (1.2) | 0.34 | .29 |
aRating scale: 1 = None; 2 = Minimal; 3 = Adequate; 4 = More than adequate; 5 = Extensive.
ANOVA comparing mean scores on website content ratings between the two datasets showed improvements in providing personalized advice to quit smoking (F1,68 = 3.82,
Percentage of websites offering “more than adequate” or “extensive” coverage of key topic areas in 2004 and 2007 review
2004 | 2007 |
|
|
|
1. Advise every tobacco user to quit: strong | 2 | 9 | 1.6 | .25 |
2. Advise every tobacco user to quit: personalized | 4 | 17 | 3.3 | .08 |
3. Assess readiness to quit | 9 | 13 | .32 | .67 |
4. Assist with quit plan | 17 | 39 | 3.9 | .04 |
5. Provide practical counseling | 13 | 39 | 6.1 | .01 |
6. Provide intra-treatment social support | 15 | 27 | 1.2 | .30 |
7. Recommend pharmacotherapy | 13 | 22 | .87 | .42 |
8. Arrange follow-up | 0 | 13 | 6.4 | .01 |
Enhance Motivation: | ||||
9. Relevance | 15 | 39 | 4.9 | .07 |
10. Risks | 11 | 44 | 9.6 | < .001 |
11. Rewards | 17 | 44 | 5.4 | .02 |
12. Roadblocks | 17 | 35 | 2.6 | .19 |
The goal of this study was to examine the content and quality of WATIs that are most likely to be encountered by smokers looking for online cessation assistance. We were also interested to see whether there were changes in quality between websites reviewed in the current study and those of our previous review, published in 2004 [
Results of this study indicate that while the majority of reviewed websites provide coverage for most key content areas identified as the core of smoking cessation treatment [
Providing practical counseling was the key content area that received the most coverage: all websites provided at least minimal coverage of this topic. This is not surprising given that providing practical counseling and information could be described as the core content area of tobacco dependence treatment. However, within this topic, few websites prompted users to reflect back on prior quit attempts. Such reflection is important as it helps smokers to identify triggers, situations that are high risk for relapse, and techniques they found useful and could employ again [
There was a notable lack in using the capacity of the Internet for personalization of treatment. An important part of motivating smokers to quit is to personalize information relevant to quitting. Reasons for quitting, perceived risks of continued smoking, as well as perceived benefits and barriers to quitting should all be identified by the individual to have maximal impact [
Related to this issue, the interactive capability of WATIs was generally underutilized. Across all of the key treatment components, only a minority of websites provided interactive features. The most common use of interactivity was in the area of providing intra-treatment social support, frequently in the form of chat rooms, buddy lists, and emailed support. Recommendations for pharmacotherapy were frequently interactive in nature, although limited to the administration and scoring of assessments of nicotine dependence. Perhaps the most glaring failure to leverage the capabilities of the Internet was in providing follow-up contact. Follow-up contacts can be used to motivate smokers to make a quit attempt or to reconsider cessation following slip/relapse and to provide support during difficult times while quitting [
Compared with our earlier review, the current results indicate modest improvement in the quality of coverage in key content areas. Areas showing the most improvement were giving personalized advice to quit, providing practical counseling, recommending medications to aid quitting, and enhancing motivation (discussing personal relevance of quitting, perceived risks, and roadblocks to quitting). In no case did we observe a significant decline in the quality of website content. It is encouraging that the quality of some content areas may be improving. However, there remains substantial room for further improvements. Most often, the reviewed websites provided only minimal coverage of key component areas. Only in discussions of risks and roadblocks did most websites provide more than adequate or extensive coverage, and even in these areas, few websites took advantage of the interactive capacity of the Internet to truly personalize treatment.
Results of this study should be considered in the context of several limitations. First, this review should not be considered an exhaustive analysis. In the present study, we reviewed only English-language websites. Thus, the quality of websites available in other languages remains unknown. Given that much of the world is non-English-speaking, we encourage researchers with fluency in other languages to conduct similar reviews of non-English websites. Second, comparisons between the 2004 and current review are made with a notable caveat. The search procedure used in the 2004 paper was comprehensive, including all websites retrieved in the first 10 pages of search returns. The current review used a search strategy that was designed to mimic the search pattern of most Internet users. That is, we included only those websites retrieved on the first page of search returns. It is possible that higher quality websites are more likely to be retrieved in the first page of search returns. If true, the increases in quality observed between the 2004 and current dataset may be an artifact of the search procedures rather than a reflection of a real improvement in the quality of smoking cessation websites. In a review of popular smoking cessation websites identified by survey respondents, Etter [
Results of this study indicate that the content and quality of information contained in smoking cessation websites may be improving. However, more often that not, smokers looking for assistance online will find websites that do not provide evidence-based guidance and assistance. Moreover, numerous questions remain to be answered about WATIs. Research needs to move beyond quantitative assessments of the amount and accuracy of information provided via the Web and begin to examine the qualitative nature of WATI sites and the relationships that exist between these sites and their users. For example, research is needed to determine whether there are intrapersonal (eg, age, education, health literacy, need/preference for social support) or environmental characteristics (eg, support/incentives from a health maintenance organization or employer, presence of smoking policy restrictions) that predict better or worse outcomes when using Internet-delivered cessation interventions. The Internet holds great potential to reach millions of smokers who may not otherwise seek cessation treatment. Efforts are needed to ensure that the content of Internet interventions is sound so that we can begin to understand how, for whom, and by what mechanism(s) WATIs may be effective.
Drs. Bock and Graham have worked with QuitNet.com as consultants and co-investigators on research. Therefore, the rating of the QuitNet.com website was performed by Drs. Whiteley and Stoddard.
nicotine replacement therapy
Public Health Service
Smoking Treatment Scale - Content
Smoking Treatment Scale - Rating
Web-assisted tobacco intervention