Anxiety disorders are common problems that result in enormous suffering and economic costs. The efficacy of Web-based self-help approaches for anxiety disorders has been demonstrated in a number of controlled trials. However, there is little data regarding the patterns of use and effectiveness of freely available Web-based interventions outside the context of controlled trials.
To examine the use and longitudinal effectiveness of a freely available, 12-session, Web-based, cognitive behavioral therapy (CBT) program for panic disorder and agoraphobia.
Cumulative anonymous data were analyzed from 99695 users of the Panic Center. Usage statistics for the website were examined and a longitudinal survey of self-reported symptoms for people who registered for the CBT program was conducted. The primary outcome measures were self-reported panic-attack frequency and severity at the beginning of each session (sessions 2-12).
Between September 1, 2002 and February 1, 2004, there were 484695 visits and 1148097 page views from 99695 users to the Panic Center. In that same time period, 1161 users registered for the CBT program. There was an extremely high attrition rate with only 12 (1.03%) out of 1161 of registered users completing the 12-week program. However, even for those who remained in the program less than 12 weeks we found statistically significant reductions (
Freely available Web-based self-help will likely be associated with high attrition. However, for the highly self-selected group who stayed in the program, significant improvements were observed.
Anxiety disorders are common problems that result in enormous suffering and economic costs [
Self-help therapy for anxiety disorders has been found to be effective, especially when the interventions are tailored to the individual's specific symptoms and situation and administered with a minimal amount of professional guidance and support [
There has been some research on Web-based programs designed to provide relatively generic CBT interventions for depression and anxiety [
Although the evidence for the efficacy of Web-based self-help for mood and anxiety disorders from controlled trials is encouraging, it is important to determine how such programs are utilized and to estimate their effectiveness when accessed by diverse, less well-selected groups of users under less controlled conditions. To this end, Christensen et al [
In contrast to previous reports of the efficacy of computer-assisted and Web-based interventions for anxiety in well-controlled research settings, in the present study we examined the use and effectiveness in an uncontrolled visitor population of a freely available Web-based CBT program for panic disorder.
The Panic Center [
Panic program process
Sample weekly review at session 2
As illustrated in
As an alternative to using the Web-based treatment program, users can download an Adobe version of the 12-session program and use the hard copy as a traditional self-help book. Although this option reduces the number of people using the Web-based program and options for collecting data about the use and effectiveness of the program, it is offered in the interest in maximizing the dissemination and use of the program.
The following describes some of the components in more detail.
The support group format consists of asynchronous communication (bulletin board format) between members of the support community and the moderators. Users of the support group also have access to individualized email support and advice from the moderators, who are Registered Nurses (Multimedia Appendix Slide 8).
The Web-Based Depression and Anxiety Test (WB-DAT) is a self-report screening tool for mood and anxiety disorders compatible with the DSM-IV [
WB-DAT panic disorder screener
The panic symptom diary (Panic Diary) allows users to record and track the frequency and severity of their panic attacks, their overall daily level of anxiety and depression, and their medication(s) and dose(s)
Panic diary recording form
The CBT program was designed based on current evidence for the effective components of CBT interventions for panic disorder and agoraphobia. The essential components of the CBT program include orientation to the cognitive behavioural model of panic disorder and agoraphobia, goal setting, exposure work exercises, cognitive restructuring, interoceptive exposure work, relaxation training, and information about lifestyle change and stress management (Multimedia Appendix Slide 5). Users are assigned homework to complete each week. As mentioned previously, users are at the beginning of each session asked to respond to a number of questions about their symptoms, homework and progress to date (Weekly Review, see
In order to determine the overall usage of the individual Panic Center tools, we examined log statistics regarding website usage and traffic, including overall statistics regarding the number of visitors to the website, page views, and usage of the screening test, symptom diary and support group. With respect to evaluating the effectiveness of the CBT program, we conducted a longitudinal survey examining data from the Weekly Review questions as well as the screening assessments conducted at registration and at the end of session 12.
This study was approved by the Research Ethics Board at the Centre for Addiction and Mental Health in Toronto, Ontario, in accordance with all applicable regulations. With respect to the log statistics, the number of unique visitors was determined based on IP addresses. WebTrends was used to analyze log files. No techniques were employed to analyze the log file for identification of multiple entries. With respect to the informed consent process for evaluating the use of the panic symptom diary and the WB-DAT, and effectiveness of the CBT program, users were informed of the approximate length of time of the surveys, which data are stored and where and for how long. Users were neither informed of the specific name of the investigators nor the specific purpose of the study. They were informed that “. . . anydata that is collected is cumulative. That means we compile your data with the results of others. We do not keep individual statistics and we are unable to find out who you are.” The policy also informed users that “Your information will be grouped with other peoples' information so that independent researchers can conduct research to improve the system for other people with panic disorder and agoraphobia. We will not sell e-mail identification, names or addresses to third parties.”
No personal identifying information was collected or stored. A number of specific measures were taken to protect the privacy of the participants and unauthorized access including the following:
Users do not have to provide any identifying information when they access the website or register to use any of the tools. Therefore, these are essentially cumulative and anonymous survey data.
Users are not required to provide any identifying information when they register for the WB-DAT, support group, symptom diary, or CBT program. In order to ensure anonymity, they are in fact discouraged from using their real names or email addresses. Users are explicitly asked to use a pseudonym when they use the program and are asked to create a hotmail or Yahoo account using a pseudonym so that they cannot be identified by their email address.
The design of the Panic Center strictly adheres to international laws that protect privacy. The data collection methodologies follow guidelines set forth by the Personal Information Protection and Electronic Documents Act (PIPEDA) [
Security of the database is assured by a robust firewall setup that sits at the edge of their Web network to secure the flow of data. The network operations are manned 24 hours a day, 7 days a week, and a security officer is present round-the-clock. Closed Circuit Television (CCTV) monitors all access points at the server co-location facility, Peer1 Networks [
The usability and technical functionality of the electronic data collection was rigorously tested and subjected to quality assurance (tested on multiple browsers, error checking code implemented, unit testing) before data collection began. The format of data collection was a “closed survey” posted on a website and initial contact was made on the Internet. No incentives were offered. Items were not randomized. The maximum number of items per page was 32. A completeness check was performed using JAVAscripterror checking. All questions were static and mandatory. Adaptive questioning was not used. Most questions did not allow for a not applicable response. Respondents were not able to review and change their responses. The “view rate” (as defined by Eysenbach [
The sample was a self-selected convenience sample. Cumulative anonymous logfile data were analyzed from 99695 users of the Panic Center from September 1, 2002 to February 1, 2004. In addition, we examined self-reported outcome data from 1161 people who registered for the CBT program (the Panic Program) within the same time frame.
We examined cumulative data regarding website usage (traffic) including number of visits, number of page views, number of unique visitors, and average viewing time (length of visit).
We examined cumulative data regarding usage of the WB-DAT including number of tests completed, number of males and females completing the test, average number of diagnoses per user and the relative frequency for users meeting screening criteria for the anxiety disorders, major depressive disorder (MDD), and dysthymia. In addition, we asked users what they intended to do with their screening test results. We examined cumulative data regarding use of the symptom diary including number of registered users and their gender. We examined cumulative data regarding usage of the support group including number of visitors, number of registered members, and number of posts.
When individuals registered for the Panic Program, they were asked a number of questions about their current symptoms, including questions about the frequency and intensity of their panic attacks, as well as the degree to which their symptoms interfered with their daily lives. In addition, users were asked to indicate whether they were using the program on their own or in collaboration with a health care professional. At the beginning of each session, users were asked a number of questions regarding their symptoms, homework and progress to date (Weekly Review). At the end of session 12 users were asked to respond to a number of questions regarding the frequency and severity of their panic attacks as well as the degree to which their symptoms interfered with their daily lives. Finally, users are asked to complete the WB-DAT at the time they register for the program as well as at the end of session 12.
We evaluated the effectiveness of the Panic Program in three ways. First we used the Weekly Review data to compare the reported frequency and severity of panic attacks at the beginning of sessions 2, 4, 6, 8, 10, and 12. Second, we compared data on the degree to which users' panic attacks interfered with their daily lives at the time they registered for the program and at the end of session 12. Third, we compared users' WB-DAT data at registration and at the end of session 12 to determine the number of users who met screening criteria for DSM-IV Axis I diagnoses at the time they registered for the program compared to the end of session 12. Dimensional data regarding frequency and severity of panic attacks and interference in daily life were analyzed using paired-samples
Between September 1, 2002 and February 1, 2004, there were 484695 visits and 1148097 page views from 99695 unique visitors to the Panic Center. The average length of a visit was 13 minutes and 11 seconds (SD [standard deviation] 4 minutes, 21 seconds). There were 28123 unique visitors to the Panic Program, WB-DAT, and Panic Diary and 356134 page views of those features.
Between September 1, 2002 and February 1, 2004, 15269 users completed the WB-DAT.
Number of screening diagnoses criteria met by users of the WB-DAT
|
|
|
Total males | 5075 | 33.24 |
Total females | 10194 | 66.76 |
Total tests with no diagnosis | 1933 | 12.66 |
Total tests with 1 diagnosis | 3691 | 24.17 |
Total tests with 2 diagnoses | 2731 | 17.89 |
Total tests with 3 diagnoses | 2237 | 14.65 |
Total tests with 4 diagnoses | 1890 | 12.38 |
Total tests with 5 diagnoses | 1474 | 9.65 |
Total tests with 6 diagnoses | 1056 | 6.92 |
Total tests with 7 diagnoses | 257 | 1.68 |
Total tests with 8 diagnoses | 0 | 0 |
Number of users meeting screening criteria on the WB-DAT
|
|
|
No diagnosis | 1933 | 12.66 |
Major depressive disorder | 2021 | 13.24 |
Dysthymic disorder | 4107 | 26.90 |
Generalized anxiety disorder | 5891 | 38.38 |
Obsessive compulsive disorder | 2504 | 16.40 |
Panic disorder with agoraphobia | 4360 | 28.55 |
Panic disorder without agoraphobia | 254 | 1.66 |
Agoraphobia without a history of panic disorder | 2971 | 19.46 |
Social phobia (generalized subtype) | 3643 | 23.86 |
Social phobia (nongeneralized subtype: public speaking) | 3525 | 23.09 |
Specific phobia | 40 | 00.26 |
Post-traumatic stress disorder | 3707 | 24.28 |
Acute stress disorder | 44 | 00.29 |
Out of 15229 users, 6687 (43.79%) responded to the survey. Of these 1388 (20.76%) reported that they intended to share the results with their doctor; 2517 (37.64%) reported that they were going to think about sharing the results with their doctor; 777 (11.62%) reported that they were not going to share the results with their doctor; 229 (3.42%) reported that they were health care professionals reviewing the test; and 1776 (26.56%) had “no comment.” Of the total number of users who completed the screening test, 4003 (26.21%) printed their results (Final Report), 1676 (10.97%) emailed their results to themselves, and 198 (1.29%) emailed their results to a health care professional.
Between September 1, 2002 and February 1, 2004, 493 (357 [72.41%] female and 136 [27.59%] male) users registered to use the panic symptom diary (Panic Diary) without also registering for the CBT program. During the same time period, 1451 users registered for the online support group and there were a total of 6664 posts and 75622 visitors. On average, each post was viewed by 8.81 (SD 2.34) visitors.
Between September 1, 2002 and February 1, 2004, 856 (73.90%) females and 305 (26.1%) males registered for the Panic Program. Out of 1161, 126 (11%) reported that they were using the program “with a health care professional” and 1065 (92%) reported that they were using it “on their own.” In addition, 190 users reported that they were “a health care professional reviewing the program.” Their data were excluded from further analyses. The Panic Program in booklet form was downloaded by 1059 users.
Number of users who completed each session of the 12-session CBT program
|
|
|
Session 1 | 1161 | N/A |
Session 2 | 525 | 45.22 |
Session 3 | 152 | 28.95 |
Session 4 | 145 | 95.39 |
Session 5 | 91 | 62.76 |
Session 6 | 46 | 50.55 |
Session 7 | 39 | 84.78 |
Session 8 | 30 | 76.92 |
Session 9 | 28 | 93.33 |
Session 10 | 22 | 78.57 |
Session 11 | 16 | 72.72 |
Session 12 | 12 | 75.00 |
The primary outcome measure for the effectiveness of the Panic Program was user's self-report of panic attack frequency and severity at the beginning of each session (sessions 2-12). At the beginning of each session users were asked to report the number of panic attacks they had experienced per day for the previous week and the average intensity of those panic attacks on a scale from 0 to 10 with 0 being “no panic” and 10 being as intense as the “worst attack ever”
Average number of panic attacks per day in the past week
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Average intensity of panic attacks in the past week
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Only 12 users completed all outcome measures, including the WB-DAT. At session 1, those 12 individuals met criteria for an average of 1.42 (SD 0.90) DSM-IV Axis 1 Disorders according to the screener. At session 12, they met criteria for an average of 0.42 (SD 0.79) disorders (
At registration and at the end of session 12, users were asked a number of questions, including a question about the degree to which their panic attacks interfered with their normal daily lives on a 0 to 4 scale with 0 being
In response to the survey at the end of session 12, 12 out of 12 (100.00%) users reported that since challenging the Panic Program they were challenging their anxious thoughts, 11 out of 12 (91.67%) reported that they were getting better at setting goals and designing exposure plans, 12 out of 12 (100.00%) reported that since starting the Panic Program they had gained confidence in their ability to challenge their fears and win, and 12 out of 12 (100.00%) reported that they believed that their hard work was paying off. Out of 12 users, 10 (83.33%) reported that they used the Support Group and 10 out of 10 (100%) rated the Support Group as “extremely helpful.”
This study evaluated the patterns of use and effectiveness of a Web-based self-help program for panic disorder and agoraphobia. We found that the website is popular and well utilized. Users tend to visit the website several times and spend considerable time on the website. With respect to the goal of increasing collaborative disease management and promoting communication between consumers and health care professionals, it would appear that the website is being used for that purpose. For example, approximately 50% of users who complete the WB-DAT report that they either intend to share the results with a health care professional or are considering doing so, and approximately 10% of users reported that they were using the CBT program in collaboration with a health care professional. A small but noteworthy percentage of people who registered to use the WB-DAT and CBT program, and those who downloaded the print version of the CBT program identified themselves as health care professionals.
Among the interesting findings from this study is the fact that a fairly high proportion of users who completed the WB-DAT met criteria for one or more anxiety disorders. It appears that users of the website are likely people who are self-selected because they are suffering from some type of anxiety disorder and perhaps especially panic disorder or agoraphobia. It is also interesting that most support group users were passive visitors and viewers as opposed to users who post information.
The data regarding the usage and effectiveness of the CBT program are also interesting. Although many people used the program for a few weeks, only a few used it for the entire 12 sessions. However, consistent with the literature [
It is important to note that these data were collected in an uncontrolled fashion. In contrast to previous reports of controlled trials of computer and Web-based interventions, we analyzed cumulative anonymous data from a freely available program. In addition, the sample was not demographically well characterized. In order to ensure anonymity, only minimal demographic data were collected. Because this was a longitudinal design with no control group we do not know whether the highly self-selected group of users who stayed in the program would have become better also without the intervention.
The most notable problem is the high attrition rate, which is consistent with other research on self-help interventions [
The high attrition rate may also be caused by the option of downloading a PDF file of the entire Panic Program. Given that 1161 users registered for the program and 1159 users downloaded the PDF version, it seems likely that many users preferred to read from the hard copy. They may have stopped using the Web-based program and their data regarding their usage of the program was therefore lost. However, they may have continued to use the hard copy to some effect. It also may be that many people choose to use self-help resources in a nonlinear manner.
The results of this study are consistent with the results of recent research demonstrating the efficacy of Web-based self-help for panic disorder [
In summary, despite the high attrition rate, these data suggest that freely available Web-based self-help for panic disorder can be effective for self-selected individuals. Such a result is interesting given the cost-effectiveness of Web-based treatments compared to conventional psychotherapeutic treatment and the potential for Web-based interventions to reach people in need [
Additional Screenshots
Dr. Farvolden has acted as a paid consultant to Van Mierlo Communications Consulting Inc., Toronto, ON, Canada, the owner of the Panic Center content and software.
Cognitive behaviour therapy
Major Depressive Disorder
Web-based depression and anxiety test