This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Internet-based guided self-help is efficacious for panic disorder, but it is not known whether such treatment is effective for milder panic symptoms as well.
To evaluate the effectiveness of Don’t Panic Online, an Internet-based self-help course for mild panic symptoms, which is based on cognitive behavioral principles and includes guidance by email.
A pragmatic randomized controlled trial was conducted. Participants (N=126) were recruited from the general population and randomized to either the intervention group or to a waiting-list control group. Inclusion criteria were a Panic Disorder Severity Scale-Self Report (PDSS-SR) score between 5-15 and no suicide risk. Panic symptom severity was the primary outcome measure; secondary outcome measures were anxiety and depressive symptom severity. Measurements were conducted online and took place at baseline and 12 weeks after baseline (T1). At baseline, diagnoses were obtained by telephone interviews.
Analyses of covariance (intention-to-treat) showed no significant differences in panic symptom reduction between groups. Completers-only analyses revealed a moderate effect size in favor of the intervention group (Cohen’s d=0.73,
The Internet-based guided self-help course appears to be ineffective for individuals with panic symptoms. However, intervention completers did derive clinical benefits from the intervention.
Nederlands Trial Register: NTR1639; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=1639 (Archived by WebCite at http://www.webcitation.org/6ITZPozs9).
Panic disorder (PD) with or without agoraphobia is a prevalent anxiety disorder associated with substantial loss of quality of life for the patient and considerable costs to society [
For treatment, PD can be effectively treated with psychological or drug therapy [
Internet-based guided self-help has shown to be an efficacious treatment of PD as well, with a large effect size (Hedge’s
The current study is a pragmatic randomized controlled trial (RCT) of the effectiveness of Don’t Panic Online in reducing panic and anxiety symptoms among participants with subclinical and mild clinical PD. We postulate a difference in effect between Don’t Panic Online and a waiting-list control group.
We conducted a pragmatic RCT with 2 arms: (1) Internet-based guided self-help, and (2) a waiting-list control group (see subsequent description). The Medical and Ethical Committee of VU University Medical Center approved the study protocol, which is described in greater detail elsewhere [
We included participants aged 18 and older, with subclinical PD or clinical PD with relatively mild symptom severity, who had access to the Internet. Any individuals who were at risk of suicide were excluded. Subclinical or mild PD was defined as having a score ranging from 5 to 15 on the Panic Disorder Severity Scale-Self Report (PDSS-SR) [
Previous RCTs of Internet-based self-help interventions for panic symptoms showed large between-group effect sizes [
Participants were recruited from the general population. Most of those who applied for participation did so after reading about this study in the health section of an online newspaper. Additional online recruitment was conducted by means of a Facebook advertising campaign and by posting messages on panic-related or anxiety-related message boards. This was supplemented by offline recruitment by means of advertisements in national newspapers and articles in local newspapers. Interested individuals were directed to a study website, where they could find information about participation and a downloadable informed consent form. The application procedure involved printing and signing the informed consent form, then sending this to the research team (either as a physical document, by conventional mail, or as a scanned document attached to an email).
Consenting applicants were sent an email with a link to the online questionnaires. The baseline (T0) questionnaires included the screening questionnaires for inclusion. Any participants who reported severe panic symptoms or who were at risk of suicide were sent an automatic message advising them to contact their general practitioner and/or to visit a website for suicide prevention. This website [
Don’t Panic Online is a guided, Internet-based, individual, self-help course, based on cognitive behavioral therapy principles. The course consists of 6 sessions in which the participants learn to control their panic symptoms by applying various cognitive and behavioral techniques and skills. The course’s content is described in more detail elsewhere [
Participants in the control group received access to Don’t Panic Online after completing the T1 measurement (12 weeks after T0). While waiting, they had access to an information website about the symptoms of panic and agoraphobia. This website included advice to contact a general practitioner in case the participant had further questions about panic symptoms and its treatment. All participants in the control group and the intervention group were free to seek any (additional) help they might require.
The following variables were measured: demographic data,
The T0 measurement started with demographic questions. These included age, gender, place of birth, marital status, education level, physical health, and previous mental health diagnoses.
The Composite International Diagnostic Interview (CIDI) 12-month prevalence [
The severity of current panic symptoms was measured using the PDSS-SR. The PDSS, which was originally designed as a face-to-face interview for both research and clinical practice [
Anxiety symptoms in general were measured using the Beck Anxiety Inventory (BAI) [
Depressive symptoms were measured using the Center for Epidemiologic Studies Depression scale (CES-D) [
Suicide risk and suicidal ideation were measured using the specific section of the Mini-International Neuropsychiatric Interview (MINI) [
An indication of health care services usage during the past month was obtained using Part I of the Trimbos and Institute of Medical Technology Assessment Questionnaire on Costs Associated with Psychiatric Illness (TiC-P) [
Finally, the T1 battery of online questionnaires included open questions concerning the participant’s subjective experience with Don’t Panic Online and reasons for not finishing the program. These questions were only administered to the intervention group.
Firstly, means and standard deviations were calculated for age and symptom severity of panic, anxiety, and depression. Any differences in symptom severity between the intervention group and control group were expressed in terms of Cohen’s
Between-group effects at T1 were calculated using analyses of covariance (ANCOVA), controlling for pretreatment scores. Instead of
The data were analyzed in agreement with the intention-to-treat (ITT) principle. Missing data at T1 were imputed by multiple imputation, in which all variables except for nominal variables (ie, age, education level, clinical diagnoses, and symptom severity on all measures at T0 and T1) were included as predictors. Ten datasets were generated and analyses were performed using pooled data. Compared with single imputation methods, multiple imputation generates a more conservative estimate of the sample standard error [
Of 368 applicants who applied and sent in informed consent forms, 126 were included in the study. See
The posttreatment measurement was completed by 73 participants (57.9%). There was no significant difference between the measurements and characteristics of these 73 study completers and those of the 53 participants who were lost to follow-up. However, within the intervention group, study dropouts were less likely to have completed lessons 1 to 4 of the course (χ2
1 = 15.1,
Of the 63 participants in the intervention group, 60 (95%) started lesson 1, whereas 3 participants did not log in at all (
After multiple imputation, ANCOVAs showed no significant difference in panic symptom severity at T1 between groups as measured by the PDSS-SR (
At T1, and with missing values imputed, 24 participants (38%) in the intervention group and 13 (20%) in the control group had PDSS-SR scores of less than 5 (ie, symptom free). This difference did not reach significance (χ2 = 5.7,
Sensitivity analyses with the EM algorithm gave slightly different results. There was no significant effect between groups on the primary outcome measure (PDSS-SR:
Baseline characteristics of participants.
Characteristics | Total sample |
Intervention group |
Control group |
Difference at baseline (Cohen’s |
|
|
|
|
|
|
|
|
Age, mean (SD) | 36.6 (11.4) | 36.7 (12.2) | 36.4 (10.7) |
|
|
Female, n (%) | 85 (67.5) | 44 (69.8) | 41 (65.1) |
|
|
Born in the Netherlands, n (%) | 115 (91.3) | 57 (90.5) | 58 (92.1) |
|
|
Living alone, n (%) | 50 (39.7) | 23 (36.5) | 27 (42.9) |
|
|
High education,a n (%) | 63 (50.0) | 30 (47.6) | 33 (52.4) |
|
|
Physical health problems, n (%) | 9 (7.1) | 5 (7.9) | 4 (6.3) |
|
|
Previously diagnosed with a mental disorder, n (%) | 47 (37.3) | 22 (34.9) | 25 (39.7) |
|
|
|
|
|
|
|
|
PD with agoraphobia | 61 (49.2) | 30 (47.6) | 31 (49.2) |
|
|
PD without agoraphobia | 36 (29.0) | 17 (27.0) | 19 (30.2) |
|
|
Agoraphobia without PD | 17 (13.7) | 10 (15.9) | 7 (11.1) |
|
|
GAD | 11 (8.9) | 5 (7.9) | 6 (9.5) |
|
|
Social phobia | 78 (62.9) | 39 (61.9) | 39 (61.9) |
|
|
PTSD | 16 (12.9) | 4 (6.3) | 12 (19.0) |
|
|
MDD | 53 (42.7) | 27 (42.9) | 26 (41.3) |
|
|
|
|
|
|
|
|
Panic (PDSS-SR) | 8.9 (3.0) | 8.8 (3.2) | 9.1 (2.8) | 0.12 |
|
Anxiety (BAI) | 24.9 (10.8) | 23.7 (10.2) | 26.0 (11.3) | 0.22 |
|
Depression (CES-D) | 20.8 (9.0) | 20.0 (9.1) | 21.6 (9.0) | 0.18 |
aDefined as the equivalent of a bachelor’s degree or higher.
bPercentages add up to more than 100% due to comorbid diagnoses.
cMissing data of 2 participants (n=124).
Use of care in the past month.
Care use | T0, n (%) | T1,a n (%) | ||
|
Intervention group |
Control group |
Intervention group |
Control group |
Visited general practitioner | 27 (43%) | 31 (49%) | 2 (13%) | 12 (31%) |
Visited psychologist or psychiatrist | 23 (37%) | 17 (27%) | 5 (31%) | 14 (36%) |
Visited other professional health care giver | 18 (29%) | 25 (40%) | 3 (19%) | 14 (36%) |
Used antidepressants, sedatives, or sleeping pills | 20 (32%) | 23 (37%) | 7 (44%) | 13 (33%) |
aDifferences within groups and between groups did not reach significance.
Flow of participants through the study.
Reasons why participants did not finish Don’t Panic Online within 12 weeks (n=30).
Reason for discontinuation | na |
Time constraint (too busy or need more time) | 13 |
Life events (pregnancy, loss, family issues) | 5 |
Symptoms too severe to do assignments | 5 |
Found other therapy | 4 |
Content not applicable | 3 |
Spontaneous recovery | 2 |
Adverse effect | 1 |
More guidance needed | 1 |
Lack of structure | 1 |
Lessons too slow | 1 |
Not motivated | 1 |
aNumbers do not add up to 30 because 2 participants did not give reasons and others gave several.
Differences between groups at T1, intention-to-treat (N=126).
Measure | Group, mean (SD)a | Between-groups effecta | ||||
|
Intervention group |
Control group |
Cohen’s |
ANCOVAb | ||
|
|
|
|
|
|
Partial |
PDSS-SR | 5.8 (4.9) | 7.3 (4.9) | 0.30 (–0.91, 1.51) | –1.17 | .25 | .023 |
BAI | 17.0 (12.7) | 22.0 (12.7) | 0.39 (–2.74, 3.53) | –1.71 | .09 | .027 |
CES-D | 16.4 (12.3) | 21.1 (12.1) | 0.39 (–2.59, 3.42) | –1.56 | .12 | .034 |
aMissing data imputed by multiple imputation.
bControlling for symptom severity at T0.
cDegrees of freedom not provided due to multiple imputation.
Those participants in the intervention group who had completed the first 4 lessons (or more) of the course (n=17) were included in the completers-only analyses. These completers cannot be all considered to have completed the intervention, but after 4 lessons, participants can be considered to have experienced most of the content of the intervention. Sixteen of the 17 participants in the intervention group who had completed the first 4 lessons also filled in T1 questionnaires. Accordingly, there were 16 completers in the intervention group. These 16 individuals did not significantly differ from the noncompleters in the intervention group at T0 in terms of age, education, clinical diagnosis, and symptom severity. Control group completers were those who filled in T1 (n=39).
The ANCOVA showed significant differences between the intervention group completers and control group completers with regard to panic symptom severity at T1 (
Ten (68%) of the intervention completers and 8 (21%) of the control group completers had a PDSS-SR score of less than 5 at T1, which is a significant difference (χ2
1 = 9.1,
Lastly, health care service usage rates did not differ either within or between groups (see
Neither ITT nor completers-only analyses showed differences on any outcome measure between participants with and without clinical PD.
Differences between groups at T1, completersa (n=55).
Measure | Group, mean (SD) | Between-groups effect | ||||
|
Intervention group |
Control group |
Cohen’s |
ANCOVAb | ||
|
|
|
|
|
|
Partial |
PDSS-SR | 4.6 (3.3) | 7.5 (4.2) | 0.73 (–0.60, 2.32) | –2.60 | .01 | .115 |
BAI | 15.6 (13.4) | 22.6 (11.2) | 0.60 (–2.93, 7.15) | –2.37 | .02 | .098 |
CES-D | 12.1 (8.5) | 21.6 (11.0) | 0.94 (–2.50, 5.10) | –2.52 | .02 | .109 |
aControl group completers are those who provided posttreatment data. Intervention group completers are those who provided posttreatment data and completed at least lesson 4.
bControlling for symptom severity at T0.
This study showed that the Internet-based, guided, self-help intervention Don’t Panic Online was not effective in individuals with panic symptoms. Completers-only analyses did show moderate to large effect sizes between groups in favor of the intervention group. Adherence to the treatment was low. An analysis of the data using a less conservative imputation method revealed significant effects between groups in terms of the scores for general anxiety and depressive symptoms, but not for panic symptoms. Overall, the results show that Don’t Panic Online could be efficacious for intervention completers, but that it is not generally effective.
A meta-analysis revealed that the psychological treatment (offline and online) of full-blown PD is highly effective compared to a waiting-list control group, with a mean effect size of
For study design and intervention, our study is comparable with the trial of Meulenbeek et al [
Previous studies that compared Internet-based guided self-help for panic symptoms with a control group showed an overall effect size of Hedge’s
There are several differences between our study and previous studies that may have had an impact on adherence. Firstly, all participants in our trial were free to use medication and find other treatment. Some may have found other help and decided to quit Don’t Panic Online. Secondly, our participants reported difficulties accessing the website. Thirdly, previous researchers had more telephone contact with their participants [
In summary, both clinical effect and treatment adherence were lower in our study than in previous studies of Internet-based self-help interventions and the Don’t Panic group course. The differences in sample characteristics between our study and previous trials could indicate that Internet-based interventions for panic symptoms are efficacious, but they may not be effective for all individuals seeking help for panic symptoms.
When interpreting our results, several limitations should be taken into account. One limitation of this study is nonresponse at the posttreatment measurement. For a large proportion of participants, it is unknown whether their panic symptoms increased, decreased, or remained stable. These missing values were estimated by multiple imputation. Although this can be considered a conservative imputation method, it is unlikely that the imputed values greatly underestimate the intervention effect. This is because many of the participants who did not respond at T1 also left the intervention after 1 or 2 sessions, and are, therefore, unlikely to have gained much benefit from it. A second limitation is that the intervention completers are small in number and may not be representative of the intervention group as a whole, even though there did not appear to be significant differences between completers and noncompleters. The comparison of this select group with the control group, for completers-only analyses, should be interpreted with caution. Thirdly, the control group could have had gained some benefit from the information website, which could have decreased the difference between T1 mean scores of the intervention group and control group. If that is the case, our study proved that Don’t Panic Online has, in general, no added value compared with an information website and our conclusion would remain the same. A fourth limitation is the lack of a follow-up measurement. It is not known whether the participants in either the intervention group or the control group showed any further improvement over the subsequent months to a year. Finally, all continuous measures were obtained by online self-report. The PDSS-SR could potentially yield lower mean scores than the PDSS interview [
Although previous research indicates that Internet interventions can be an efficacious treatment of panic symptoms, our results may suggest that a linear program targeting specific symptoms is not always effective. As our study and others have shown, panic symptoms generally coincide with comorbid symptoms. Therefore, transdiagnostic and tailorable interventions could be a future direction of Internet-based treatment of panic. Internet-based transdiagnostic self-help programs, tailored to the anxiety and/or depressive symptoms of the participant, show promising results in terms of the treatment of panic and other common mental disorders [
Future research could focus on identifying those groups for whom Internet-based self-help interventions are effective, for example, by means of predictor and mediator analyses. Further research is also needed to investigate ways of boosting treatment adherence to Don’t Panic Online, of making it a feasible intervention for mild to moderate panic symptoms, and perhaps of modifying it to become more tailored and transdiagnostic in nature. This was the first study of Internet-based guided self-help for mild panic symptoms and our study needs to be replicated before we can draw any definitive conclusions. Lastly, although the efficacy of Internet-based guided self-help interventions has been established in several studies, it should be encouraged to conduct more pragmatic RCTs to examine the effectiveness.
CONSORT-EHEALTH checklist V1.6.2 [
analysis of covariance
Beck Anxiety Inventory
Composite International Diagnostic Interview
Expectation Maximization
intention-to-treat
major depressive disorder
Mini-International Neuropsychiatric Interview
panic disorder
Panic Disorder Severity Scale-Self Report
randomized controlled trial
baseline
posttreatment assessment (12 weeks after T0)
This study is funded by the Trimbos-institute and VU University Amsterdam.
None declared.