Background: Previous studies provide evidence for the effectiveness of web-based interventions for panic disorder with and without agoraphobia. Smartphone-based technologies hold significant potential for further enhancing the accessibility and efficacy of such interventions.
Objective: This randomized controlled trial aims to evaluate the efficacy of a guided, hybrid web-based training program based on cognitive behavioral therapy for adults with symptoms of panic disorder.
Methods: Participants (N=92) with total scores in the Panic and Agoraphobia Scale ranging from 9 to 28 were recruited from the general population and allocated either to a hybrid intervention (GET.ON Panic) or to a wait-list control group. The primary outcome was the reduction in panic symptoms, as self-assessed using a web-based version of the Panic and Agoraphobia Scale.
Results: Analysis of covariance-based intention-to-treat analyses revealed a significantly stronger decrease in panic symptoms posttreatment (F=9.77; P=.002; Cohen d=0.66; 95% CI 0.24-1.08) in the intervention group than in the wait-list control group. Comparisons between groups of the follow-up measures at 3 and 6 months yielded even stronger effects (3-month follow-up: F=17.40, P<.001, Cohen d=0.89, 95% CI 0.46-1.31; 6-month follow-up: F=14.63, P<.001, Cohen d=0.81, 95% CI 0.38-1.24).
Conclusions: Hybrid web-based training programs may help reduce the symptoms of panic disorder and hence play an important role in improving health care for patients with this debilitating disorder.
Trial Registration: German Clinical Trial Register DRKS00005223; https://tinyurl.com/f4zt5ran
International Registered Report Identifier (IRRID): RR2-10.1186/1745-6215-15-427
With a 12-month prevalence of 1.8% among adults, panic disorder is one of the most common anxiety disorders [, ]. Subthreshold cases, defined as significant panic symptoms that fail to meet full criteria, have been estimated to be just as prevalent [ , ] and have been shown to predict the development of full panic disorder as well as other mental disorders, such as generalized anxiety disorder or major depression [ ]. Effective treatments for panic disorder and associated agoraphobic symptoms include pharmacotherapy and cognitive behavioral therapy (CBT) [ - ]. Unfortunately, many individuals still lack access to evidence-based treatments because of the limited availability of clinicians or fear of stigmatization [ - ].
Technology-based psychological interventions that use the internet provide low-threshold access to evidence-based mental health care. Recent outcome studies [- ], meta-analyses, and reviews [ - ] provide ample evidence that internet-based interventions based on cognitive behavioral therapy principles (iCBT) are effective in treating panic disorder.
Owing to their ability to bridge distances between patients and therapists, good cost-efficacy and low-threshold iCBT have great potential to facilitate access to evidence-based interventions [, ]. However, the current dominance of desktop-based iCBT in research and health services neglects the dramatic shift in user preferences toward the use of smartphones [ ]. Moreover, smartphones accompany their users wherever they go, thereby providing an excellent opportunity for ecological momentary assessment of relevant health information [ - ]. Furthermore, smartphones allow the use of ecological momentary interventions to be delivered in the real world and real time, ideally at the very moment the intervention is needed [ ]. Considering the rapid growth and potential of mobile technology, surprisingly little research has been conducted to clarify the benefits of using smartphones as stand-alone or add-on interventions [ ]. Available data often come from studies criticized for poor-quality interventions [ ], and many interventions currently available have not been evaluated at all [ - ].
The few currently available studies provide preliminary evidence for the efficacy of smartphone-based interventions for the symptoms of anxiety disorders. For example, in a meta-analysis on the efficacy of transdiagnostic eHealth interventions that integrated mobile technologies, Firth et al  showed that such interventions can significantly reduce overall anxiety (Hedge g=0.45). A recent study by Christoforou et al [ ] evaluated the efficacy of an app for agoraphobic symptoms in comparison with a stress reduction app. Although there was a significant pre- to posttest effect for the interventions (Panic and Agoraphobia Scale [PAS] difference −5.97; 95% CI −8.49 to −3.44), no significant differences between the interventions were observed.
Despite these promising findings, it is important to acknowledge that mobile apps also have some disadvantages with regard to usability issues. For example, elaborate writing tasks, a typical component of iCBT interventions, are difficult to complete on a small screen with a smartphone touchpad. Moreover, cellphones are typically used for short time intervals and often while performing other tasks. This is problematic, as working toward health-promoting changes often requires more sustained and focused effort [, ]. Therefore, it can be argued that hybrid interventions that combine the advantages of both desktop and mobile technology should be superior to exclusively desktop- or mobile-based approaches. In hybrid interventions, the mobile component can be used to monitor symptoms and cue exercises in the patient’s natural environment, whereas the desktop component provides text- and video-based psychoeducation and facilitates elaborate writing tasks.
Despite the obvious advantages of hybrid interventions, the literature on their efficacy is still scarce. In a transdiagnostic approach, Proudfoot et al  showed that the delivery of CBT using a combination of mobile app and desktop-based technology was effective in reducing symptoms of anxiety disorders (Cohen d=0.47) compared with a wait-list control (WLC) condition. Furthermore, in a study evaluating the combination of Acceptance and Commitment Therapy and a smartphone app for participants with panic disorder or social phobia, Ivanova et al [ ] found no significant effect on panic symptom severity reduction. At this point, no study has been published on the efficacy of hybrid iCBT interventions for panic disorder. To fill this gap in the literature, this study aims to evaluate the efficacy of a newly developed hybrid iCBT training program for individuals with symptoms of panic disorder. Owing to the legal restrictions on remote treatment (Fernbehandlungsgesetz) [ ], we use the term online training program for the intervention format instead of the term online therapy, which is more commonly used in the literature.
To evaluate the efficacy of a hybrid web-based training program for panic disorder (with and without agoraphobia), we conducted a prospective, two-arm randomized trial, in which 92 participants with significant symptoms of panic disorder were randomly allocated either to the GET.ON Panic intervention group (IG) who received the training program immediately or to the WLC group who received the training program 6 months after randomization. For randomization, we used the automated computer program DatInf RandList version 1.2 (DatInf GmbH). The allocation was stratified for clinical or subclinical symptomatology as well as the presence or absence of agoraphobia in the order of incoming informed consent. To include equal numbers of participants in each group, we used block randomization (n=2 per block). The staff conducting the diagnostic interviews and observer ratings were blinded to the participants’ randomization statuses. The participants could participate in the training program with a pseudonym of their own choice. Ethical approval for this study was obtained from the Ethical Committee of the University of Marburg (registration number: 2013-23 K). The study was preregistered with the German Clinical Trial Register (registration number: DRKS00005223). The study protocol was submitted for publication before randomization .
Participants and Recruitment
The study participants were recruited from the general population between August 2013 and October 2014. Announcements in newspapers, support groups, and social media, such as Facebook, guided interested individuals to the web-based health center website of our research group, where they could apply on the web to participate in the study. Applicants were asked to complete a web-based questionnaire assessing the following inclusion criteria: (1) experiencing mild-to-moderate panic symptoms as assessed by the PAS (score range: 9-28) [, ], (2) being aged ≥18 years, (3) having panic as the primary concern for seeking help, (4) having internet and smartphone access with minimum system requirements of iPhone (TM) 3GS (Apple Inc; iOS 6 and iOS 7) or a comparable Android device (Android 2.3 or newer), and (5) providing their informed consent.
The exclusion criteria were as follows: (1) receiving current psychological help for anxiety problems or being on a wait-list for psychotherapy; (2) having physical health problems that were assessed via a self-report that prevents participants from engaging in self-exposure, as recommended by the German guideline for treating people with panic disorder and agoraphobia ; (3) currently having posttraumatic stress disorder or psychotic or dissociative disorders assessed via self-report and clinical interview; and (4) having current suicidality, as assessed by a score above 1 on item 9 of the Beck Depression Inventory II (BDI-II) [ , ] and question A9 of the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition Axis I Disorders (SCID-I) [ ]. In the event that potential participants were excluded because of suicidal ideation or intention, they were given information about further help according to an established suicide protocol. All excluded participants were contacted via email and provided with information regarding where they could obtain appropriate help.
Participants in the treatment condition received the GET.ON Panic treatment, which is a hybrid (ie, desktop-based and smartphone-based), iCBT-based self-help intervention for treating symptoms of panic disorder . Participants were advised to log on to the training platform, which was provided by the technical partner Minddistrict GmbH on a weekly basis and consecutively work through the following sessions: (1) psychoeducation, (2) interoceptive exposure, (3) in vivo exposure, (4) cognitive restructuring—introduction, (5) cognitive restructuring—extension, and (6) relapse prevention. In addition, participants were instructed on the complementary use of the GET.ON Panic app [ ]. The app supported participants in (1) completing their homework assignments (eg, keeping an anxiety diary); (2) planning, conducting, and evaluating interoceptive and in vivo exposure tasks; and (3) performing relaxation exercises ( ).
After every session, participants received written feedback from a trained coach based on a coaching manual developed by members of our research group to ensure a standardized procedure of coaching (the manual is available on request). The guidance focused on increasing motivation and adherence throughout the training progress, rather than providing individual therapeutic advice. The average feedback took about 20-30 min. Coaches also sent reminders via a secure messaging system within the training platform if participants did not log on for 1 week. All coaches had a degree in psychology and were supervised by a licensed clinical psychologist.
|Week||Content and homework|
|3||In vivo exposure:|
|4||Cognitive restructuring I:|
|5||Cognitive restructuring II:|
Panic Symptom Severity and Self-Rating
The primary outcome was the severity of panic and agoraphobia symptoms, as self-assessed using the PAS (German version: Panik- und Agoraphobieskala) [, , , ]. This scale consists of 13 items separated into the subscales of panic attacks, agoraphobic avoidance, anticipatory anxiety, daily life limitations, and health concerns. For each item, participants rated the frequency of panic symptoms during the past week on a 5-point scale. Thus, the total score ranges from 0 to 52, with scores ranging from 0 to 8 indicating no clinically relevant symptoms, scores ranging between 9 and 28 indicating moderate symptoms, and a score of 29 and above indicating a severe level of symptoms. Previous studies provide evidence for the efficacy of the measures, for example, Cronbach α=.86 [ ] or α=.70 to .94 [ ]. In this study, Cronbach α for the total score was .89.
Observer-Rated Anxiety Symptoms
The Hamilton Anxiety Scale (HAM-A) [- ] was used as a complement for the self-administered anxiety scales. The scale contains 14 items, with a total score ranging from 0 to 30. Previous studies showed excellent interrater and test-retest reliabilities of intraclass correlation coefficients of 0.89-0.99 [ ]. To examine interrater reliability in this trial, we audiotaped all the observer ratings. Around one-tenth (equivalent to 28 interviews) of these audio files were rated by experienced, blinded second raters. The interrater reliability was excellent, with an intraclass correlation coefficient of 0.99.
The Agoraphobic Cognitions Questionnaire (ACQ) is a 14-item self-report questionnaire that measures agoraphobic cognitions. The total sum score of the ACQ ranged from 1 to 5. The ACQ has an internal reliability of Cronbach α of .80 [- ]. In this trial, we found a Cronbach α of .84.
Bodily sensations were measured using the Body Sensations Questionnaire (BSQ), a self-rating questionnaire with the total score ranging from 1 to 5 points. It has good internal reliability of Cronbach α of .87 [- ]. In this trial, Cronbach α was .86.
The Mobility Inventory (MI) is a questionnaire that measures agoraphobic avoidance. Participants were asked to rate common agoraphobic situations with regard to their avoidance. Each item is rated twice: once for dealing with the situation alone and once when accompanied. The total score ranged from 1 to 5 points. The internal consistencies reported in previous studies were very good, with Cronbach α of .91 (accompanied by significant others) and .94 (alone) [, , ]. In this study, Cronbach α values of the MI were .93 (accompanied) and .95 (alone).
We used the German adaption (ADS) of the Center for Epidemiologic Studies Depression Scale (CES-D) to assess depressive symptom severity. The CES-D measures 20 symptoms of depression in the previous week. The total score ranges from 0 to 60. Internal consistency has been shown to be good (Cronbach α=.89) [, ]. In this study, Cronbach α was .87.
The presence of panic disorder, any other anxiety disorder, or a current depressive episode was assessed using a telephone version of the SCID-I at the 6-month follow-up (6M-FU) assessment covering the period of the last 3 months by trained interviewers. Previous studies have shown excellent test-retest reliability between the 2 different formats, the telephone version and the face-to-face (f2f) version of the diagnostic interview (Cohen =0.84) [- ]. To determine the interrater reliability of the diagnostic interviews, we used the statistics. In a previous study, moderate interrater reliability (Cohen =0.67) was found [ ]. In this trial, all interviews were audiotaped, with 11.1% (18/162) of the interviews rated by an experienced, blinded second rater. Agreement between the 2 raters was moderate, with a Cohen of 0.51.
Quality of Life
Quality of life was measured using the 12-item Short-Form Health Survey (SF-12), which assesses 8 health domains: physical functioning, role limitations, pain, general health perception, vitality, mental health, emotional role, and social functioning. The SF-12 provides 2 summary scores for physical and mental health [, ]. In this trial, Cronbach α was .79.
We assessed user satisfaction with the Client Satisfaction Questionnaire adapted to internet-based interventions (CSQ-I) , which is based on the German version of the Client Satisfaction Questionnaire [ , ]. Statements such as “I would recommend this training to a friend, if he or she was in need of similar help” are rated on a 4-point Likert scale (ranging from 1=does not apply to me to 4=does totally apply to me). The questionnaire contained 8 items, with a total score ranging from 8 to 32. The psychometric properties were excellent with a McDonald ω of 0.93 and ω of 0.95 [ ]. In this trial, McDonald ω was 0.97.
The mobile app contains a diary for recording and monitoring panic-related symptoms, such as panic events, degree of avoidance behavior, general anxiety, and mood level on a visual analog scale (0-10). Furthermore, the app recorded the type and number of exposure exercises performed by the participant. In addition, we used the System Usability Scale (SUS) at postassessment (T2) to assess the usability of the GET.ON Panic app [, ]. The sum score ranges from 0 to 100, with a higher score indicating better usability.
Participants completed a sociodemographic questionnaire, the PAS and the Suicidality item of BDI-II at screening (T0); at baseline (T1), we assessed the PAS, the SCID-I, the HAM-A, the ACQ, the BSQ, the MI, the CES-D, and the SF-12; at postassessment (T2), we assessed the PAS, the HAM-A, the ACQ, the BSQ, the MI, the CES-D, the SF-12, the CSQ-I (only IG), and the SUS (only IG); at 3-month follow-up (3M-FU; T3), we assessed the PAS, the ACQ, the BSQ, the MI, the CES-D, and the SF-12; and at 6M-FU (T4), we assessed the PAS, the SCID-I, the HAM-A, the ACQ, the BSQ, the MI, the CES-D, and the SF-12. Diary data were continuously measured during the training period and beyond ().
|Suicidality (Item 9; BDI-IIc)||X||—||—||—||—|
|Diagnosis (SCID-Id, sections for anxiety disorders and current depressive episode)||—||X||—||—||X|
|Panic and agoraphobia severity, self-rating (PASe)||X||X||X||X||X|
|Panic and agoraphobia severity, observer-rating (HAM-Af)||—||—||X||—||X|
|Agoraphobic cognitions (ACQg)||—||X||X||X||X|
|Body sensations (BSQh)||—||X||X||X||X|
|Agoraphobic avoidance (Mobility Inventory)||—||X||X||X||X|
|Depressive symptoms (CES-Di)||—||X||X||X||X|
|Quality of life (SF-12j)||—||X||X||X||X|
|User satisfaction (CSQ-Ik)||—||—||(X)l||—||—|
|Usability of smartphone app (SUSm)||—||—||(X)||—||—|
cBDI- II: Beck Depression Inventory II.
dSCID-I: Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition Axis I Disorders.
ePAS: Panic and Agoraphobia Scale.
fHAM-A: Hamilton Anxiety Scale.
gACQ: Agoraphobic Cognitions Questionnaire.
hBSQ: Body Sensations Questionnaire.
iCES-D: Center for Epidemiologic Studies Depression Scale.
jSF-12: 12-Item Short-Form Health Survey.
kCSQ-I: Client Satisfaction Questionnaire adapted to internet-based interventions.
lOnly available for intervention group.
mSUS: System Usability Scale.
To assess treatment efficacy, the GET.ON Panic group was compared with the WLC group on all outcome measures (T2, T3, and T4) using univariate analyses of covariance (ANCOVAs) with the baseline scores as covariates. On the basis of a previous meta-analysis , we powered the study to detect an effect size of Cohen d=0.6 (1− of 80%; =.05) with intention-to-treat (ITT) at T2 as our primary level of analysis. Accordingly, a sample size of 90 was required. Cohen d [ ] was used to measure effect size. To account for covariance, we calculated Cohen d over the partial eta squared (η2). To assess a clinically reliable change of panic severity (response) on an individual level, we calculated the Reliable Change Index (RCI) as proposed by Jacobson and Truax [ ], coded participants as responders or deteriorators if their score on the PAS differed by 10.68 points on the PAS, and performed a Pearson chi-square test to compare the reliable change of the GET.ON Panic group with the WLC. Corresponding to the RCI, we calculated numbers needed to treat (NNT) score indicating how many participants must take part in order for GET.ON Panic to achieve one clinically relevant improvement. To assess remission rates, we calculated the percentage of people who had a diagnostic status of panic disorder according to the SCID-I interview at baseline (T0) and at the 6M-FU (T4) and performed a Pearson chi-square test to compare the diagnostic status of the GET.ON Panic group with the WLC covering a period of the last 3 months.
Missing data at postassessment, 3M-FU assessment, and 6M-FU assessment were performed using a Markov Chain Monte Carlo multivariate imputation algorithm (SPSS 23) with hundred estimations per missing value and all available data on outcomes, age, and gender as predictors .
Over a period of 14 months, a total of 235 individuals completed the screening questionnaire. Of these, 117 did not meet the inclusion criteria or matched one or more exclusion criteria (). Severe panic symptom severity (n=54), current psychotherapy (n=34), or physical contraindications (n=23) were the most frequent reasons for exclusion. The remaining 118 candidates were eligible to participate in the clinical interviews. Of those, 19 did not provide informed consent. After this interview, another 6 candidates were excluded because they did not have panic symptoms as their primary reason for seeking help. All excluded individuals were provided with information about applicable health care system services. The remaining 92 participants were randomly assigned to the hybrid web-based training program GET.ON Panic or the WLC condition.
Most participants were female (51/92, 55%), White (76/92, 83%), on average aged 38 years (SD 10.42), highly educated (60/92, 65%), married or in a relationship (82/92, 89%), and currently working (51/92, 55%). On the basis of the SCID interview, the most common diagnosis was panic disorder with agoraphobia (78/92, 83%). A significant number of patients (12/92, 13%) met the criteria for panic disorder without agoraphobia. Of all participants, 26% (24/92) met the criteria for at least one additional anxiety disorder, in addition to panic disorder. A small percentage (2/92, 2%) had a current major depressive episode as a comorbid condition. Most participants (58/92, 63%) reported that they had previously undergone psychotherapeutic treatment ().
Study Dropout and Compliance in Treatment
Baseline data were available for all the participants. The attrition rate was 9% (8/92) at postassessment (4/45 in the IG and 4/47 in the WLC), 18% (17/92) at the 3M-FU (10/45 in the IG and 7/47 in the WLC), and 22% (20/92) at the 6M-FU (10/45 in the IG and 10/47 in the WLC;).
On average, the number of completed sessions in the GET.ON Panic group was 5.11 (SD 1.67). All 6 sessions were completed by 73% (33/45) of the participants, 4% (2/45) of the participants completed only session 1, 11% (5/45) of the participants dropped out after session 2, 7% (3/45) of the participants were lost after session 3, 2% (1/45) of the participants stopped the training after completing session 4, and 2% (1/45) of the participants after session 5. In total, 27% (12/45) of the participants did not complete the training. The reasons for intervention dropout were mentioned for 33% (4/12) of them: lack of time, lack of motivation, lack of personal contact with the eCoach, or surgery that interfered with completing the intervention. The resting 67% (8/12) of the participants were also study dropouts, and no reasons for stopping the intervention were known because they did not complete the postassessment.
Severity of Panic Symptoms
Preliminary analyses indicated that all necessary conditions for the intended statistical analyses were met. There was a greater decrease in self-reported panic disorder symptom severity in the intervention condition than in the WLC condition (and ). With regard to the primary outcome, participants in the GET.ON Panic condition reported significantly lower (baseline-controlled) panic symptom severity at posttreatment than the WLC group (F=9.77; P=.002; partial η2=0.10; Cohen d=0.66; 95% CI 0.24-1.08). This effect became even stronger (F=17.40; P<.001; partial η2=0.16; Cohen d=0.89; 95% CI 0.46-1.31) at the 3M-FU and remained significant (F=14.63; P<.001; partial η2=0.14; Cohen d=0.81; 95% CI 0.38-1.24) at the long-term 6M-FU. The effect sizes ranged from medium to large.
With regard to observer-based ratings, ANCOVA showed a significant difference in anxiety symptoms between groups as measured by the HAM-A at postmeasurement (F=3.97; P=.049; partial η2=0.04; Cohen d=0.42; 95% CI 0.01-0.84) and at the 6M-FU (F=4.86; P=.03; partial η2=0.05; Cohen d=0.47; 95% CI 0.05-0.88) with small-to-medium effect sizes. Further analyses indicated that the findings did not significantly change when the analyses were based on the study completer instead of the ITT sample.
With regard to response, the reliable clinical changes were not significant at postmeasurement (χ22 [n=92]=2.5; P=.28; improvement: GET.ON Panic: 12/45, 27% and WLC: 7/47, 15%; deterioration: GET.ON Panic: 2/45, 4% and WLC: 1/47, 2%) or at the 3M-FU (χ22 [n=92]=5.3; P=.07; improvement: GET.ON Panic: 14/45, 31% and WLC: 6/47, 13%; deterioration: GET.ON Panic: 0/45, 0% and WLC: 1/47, 2%). However, the GET.ON Panic group was superior to the WLC in terms of the percentage of participants attaining reliable clinical change (RCI=±10.68) in panic symptom severity at the 6M-FU (χ22 [n=92]=6.0; P=.049; improvement: GET.ON Panic: 22/45, 49% and WLC: 12/47, 26%; deterioration: GET.ON Panic: 0/45, 0% and WLC: 1/47, 2%). These reliable clinical changes correspond to NNT from baseline to posttreatment of 8.49 (95% CI 3.54 to >106), at the 3M-FU of NNT=5.45 (95% CI 2.87-55.78), and at the 6M-FU of NNT=4.28 (95% CI 2.35-24.07). Regarding the long-term effect, these results indicate that 4 individuals had to participate in the GET.ON Panic training program to result in one additional individual having reliable clinical improvement in panic symptom severity.
With regard to remission rates, at baseline, nearly all participants (90/92, 98%) fulfilled the diagnostic criteria for panic disorder. At the 6M-FU, 76% of the participants (70/92, 76% GET.ON Panic group: 33/45, 73%; WLC: 37/47, 79%) agreed to the telephone-administered diagnostic interview. In total, 21% (15/70) of the participants were free of a diagnosis. There was a greater reduction in diagnoses in the GET.ON Panic group (11/33, 33%) than in the WLC group (4/37, 11%; χ21[n=70]=5.3; P=.02).
Additional Anxiety Measures
Comparing the GET.ON Panic with the WLC group on further self-rated anxiety measurements, we found stronger between-group effect sizes for agoraphobic cognitions (partial η2=0.06; Cohen d=0.51; 95% CI 0.05-0.93) and bodily sensations (partial η2=0.05; Cohen d=0.46; 95% CI 0.05-0.88) in the GET.ON Panic group than in the WLC group at posttreatment. With regard to follow-up measurements, these effects remained stable for both agoraphobic cognitions (partial η2=0.07; Cohen d=0.55; 95% CI 0.14-0.97 after 3 months and partial η2=0.05; Cohen d=0.46; 95% CI 0.04-0.87 after 6 months) and bodily sensations (partial η2=0.14; Cohen d=0.79; 95% CI 0.37-1.22 after 3 months and partial η2=0.09; Cohen d=0.66; 95% CI 0.22-1.06 after 6 months). A difference in agoraphobic avoidance between the groups could only be found when participants had to manage difficult situations when they were alone with small effect sizes (partial η2=0.05; Cohen d=0.45; 95% CI 0.04-0.86) at posttreatment and a medium effect size (partial η2=0.11; Cohen d=0.70; 95% CI 0.27-1.12) at the 6M-FU. ANCOVA did not reveal a significant difference between the groups regarding agoraphobic avoidance when participants had to manage difficult situations when they were in companionship with other people (and ).
At the postmeasurement as well as at the 3M-FU, the GET.ON Panic group showed no significant reduction in depressive symptoms compared with the WLC group (and ). However, at the 6M-FU, the depressive symptoms of the GET.ON Panic group decreased significantly with a small effect size compared with the WLC group (partial η2=0.06; Cohen d=0.49; 95% CI 0.07-0.90). The results on the quality of life scales with regard to mental health showed no reduction at postmeasurement or at the 3M-FU but a medium reduction after 6 months (partial η2=0.11; Cohen d=0.70; 95% CI 0.28-1.12). Furthermore, no differences in symptoms regarding physical aspects of quality of life were found.
App Usage and User Satisfaction
The participants of the training group (n=45) used the mobile diary on average 25.02 times (SD 19.48; range 0-56) during the 8-week training period on average (0.45 diary entries per day per participant). The repeated analysis of variance did not reveal any changes in the diary scores over a period of 8 weeks. Furthermore, they were not related to the primary outcome. The participants performed an average of 149.80 (SD 279.34; range 0-1702) interoceptive exposure exercises and 6.63 in vivo exercises (SD 17.74; range 0-113). The mean SUS score was 71.16 (SD 18.97) at posttreatment, which indicates good usability of the GET.ON Panic app. Overall, user satisfaction with the hybrid training program was high (mean 28.10, SD 5.09). For example, 91% of the participants indicated that they would recommend the training program to a friend in need.
This study aims to evaluate the efficacy of GET.ON Panic, a guided, mobile- and web-based CBT training program for adults with significant panic disorder symptoms. The results show that individuals treated with GET.ON Panic experienced a significantly greater reduction in panic disorder symptom severity than did participants in a WLC condition with between-group effect sizes of Cohen d. The findings also show that the effects were not only stable over time but even increased after the treatment was completed (Cohen d/NNT=0.66/8.49 at posttreatment vs Cohen d/NNT=0.89/5.45 after 3 months and Cohen d/NNT=0.81/4.28 after 6 months).
Comparison With Prior Work
As such, they fall well into the range of reported effect sizes in meta-analyses for internet-based interventions for panic disorder (eg, Hedge g=0.83 ; g=1.31 [ ]; Hedge g=0.83 [ ]; Cohen d=0.96 [ ]). The findings also showed that one of 3 participants in the IG had attained complete remission of panic disorder at the last assessment point, whereas this was only the case in one of 10 participants in the control group. With regard to secondary outcomes, it is of note that the 6M-FU effects on depressive symptoms are larger than the average of effects reported for psychological treatment for depression (Cohen d=0.49 vs Hedge g=0.35 [ ]). Finally, in this study, adherence rates and user satisfaction were slightly higher than those reported in previous studies (adherence: 73% vs 66%; satisfaction: 91% vs 86% [ ]).
A potential step-up could be the use of an intervention that integrates hybrid web-based training program into f2f CBT . In such blended interventions, therapists might fully exploit the potential of using the ecological momentary assessment data provided by the smartphone as well as the potential of ecological momentary interventions derived from individual case formulations and carried into the patient’s life with the help of their mobile devices [ ]. The adherence and usability rates in this study appear to be superior to what is currently reported for desktop-based iCBT interventions. This suggests that the integration of mobile components into iCBT should be a focus of future studies. The rapidly shifting use of mobile- instead of desktop-based devices underlines this [ , ].
The finding that depressive symptom severity was significantly reduced in the IG is important, as many individuals with panic disorder also have other mental health problems such as depression . As cooccurring disorders may mutually help maintain each other [ , ], it is important that comorbid conditions are treated along with the primary disorder. The positive effects of the hybrid intervention evaluated in this study on depression are consistent with the findings from CBT that successfully treat panic disorder, which also result in a reduction of depressive symptoms [ , ].
Strengths and Limitations
To our knowledge, this is the first study to examine the efficacy of iCBT training program that makes use of mobile components in a target group of people with mild-to-moderate panic and agoraphobia symptoms. One of the main strengths of this study is its solid study design, which tests a newly developed training program within a randomized controlled trial against a WLC. In addition to self-rating outcomes, we conducted clinical interviews with regard to symptom severity and changes in diagnostic status over a period of 6 months and an observer-rated anxiety outcome to validate the outcomes based on self-ratings. Furthermore, this study has high ecological validity, as participants used their own smartphones. They were supposed to interact with their smartphones as they would normally do. This may lead to a higher acceptance of and satisfaction with the GET.ON Panic training program and foster the integration of psychological interventions into the daily lives of individuals. The overall low dropout rates in this study support this assumption.
This study has several limitations that need to be considered. First, the study results cannot be generalized to all individuals with panic disorder symptoms. Participants who took part in this trial actively participated and underwent an extended eligibility procedure before the study. Many interested individuals were excluded based on the criteria defined in the study protocol. Thus, we assume that the current participants represent a more intrinsically motivated study sample and, in addition, have a higher affinity for the internet than the average individuals with panic disorder. Therefore, the external validity of this study might be limited. Second, for future treatment development, it would have been of interest to compare the hybrid intervention with both an exclusively desktop-based and an exclusively mobile-based intervention for panic disorder. However, as such a design was beyond what we could realize in this study, it would need to be used in subsequent studies. Such studies should also compare the efficacy (and cost-effectiveness) of desktop-, mobile-based, hybrid, and blended interventions with f2f therapy for panic disorder. Third, we cannot draw any conclusions on the efficacy beyond the 6M-FU assessment. Thus, future studies should evaluate the long-term effects of hybrid iCBT interventions for panic disorder.
The results of this study suggest that a significant number of individuals with symptoms of panic disorder can be helped with an intervention that is comparatively easy to disseminate and that can be used anonymously, which arguably lowers an important barrier to service utilization . However, the results also show that about two-thirds of the participants had not completely recovered after the intervention. Thus, interventions such as GET.ON Panic might best be used in a step-by-step care framework in which patients failing to attain recovery through an internet-based intervention subsequently receive more intense (and costly) interventions [ ].
This project was funded by the European Union (project number: EFRE CCI 2007DE161PR001).
LE, DL, DE, SK, HR, BF, and MB contributed to the study design. LE, DL, DE, and MB developed the hybrid web-based training program for treating people with panic disorder with or without agoraphobia (GET.ON PANIC). SK developed the app (GET.ON PANIC APP), which supervised the technical aspects of the intervention. LE performed statistical analyses. DL and SK contributed to statistical analyses. LE drafted the manuscript. MB revised the manuscript. All authors read and approved the final manuscript.
Conflicts of Interest
DL, DE, and BF are shareholders, and LE is an employee of the transfer institute (GET.ON Institute) that has usage and exploitation rights of the GET.ON Panic training program. However, at the time of developing and evaluating the app and the hybrid web-based training program GET.ON Panic, as well as drafting the manuscript, the company was not yet incorporated.
Characteristics of the study participants (N=92) allocated to online training GET.ON intervention group (n=45) and wait-list control group (WLC) (n=47) at baseline.DOCX File , 40 KB
Mean and SD values of all outcome variables at baseline, posttreatment, and at 3-month and 6-month follow-ups (intention-to-treat, N=92).DOCX File , 40 KB
Differences between groups at T2, T3, and T4 (intention-to-treat, N=92).DOCX File , 42 KB
CONSORT-eHEALTH checklist (V 1.6.1).PDF File (Adobe PDF File), 1258 KB
- Wittchen HU, Heinig I, Beesdo-Baum K. Anxiety disorders in DSM-5: an overview on changes in structure and content. Nervenarzt 2014 May 17;85(5):548-552. [CrossRef] [Medline]
- Wittchen HU, Jacobi F, Rehm J, Gustavsson A, Svensson M, Jönsson B, et al. The size and burden of mental disorders and other disorders of the brain in Europe 2010. Eur Neuropsychopharmacol 2011 Sep;21(9):655-679. [CrossRef] [Medline]
- Batelaan N, De Graaf R, Van Balkom A, Vollebergh W, Beekman A. Thresholds for health and thresholds for illness: panic disorder subthreshold panic disorder. Psychol Med 2006 Nov 01;37(2):247-256. [CrossRef]
- Kessler RC, Chiu WT, Jin R, Ruscio AM, Shear K, Walters EE. The epidemiology of panic attacks, panic disorder, and agoraphobia in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2006 Apr;63(4):415-424 [FREE Full text] [CrossRef] [Medline]
- Kinley DJ, Walker JR, Enns MW, Sareen J. Panic attacks as a risk for later psychopathology: results from a nationally representative survey. Depress Anxiety 2011 May 11;28(5):412-419. [CrossRef] [Medline]
- Bandelow B, Lichte T, Rudolf S, Wiltink J, Beutel M. S3-Leitlinie: kurzfassung der empfehlungen zur behandlung von angststörungen. Berlin: Springer; 2015:978-973.
- National Institute for Health and Care Excellence. Generalised anxiety disorder and panic disorder in adults: management. Guidance and guidelines : NICE. 2011. URL: https://www.nice.org.uk/guidance/cg113/ifp/chapter/What-treatments-should-I-be-offered-for-panic-disorder [accessed 2021-02-16]
- Sánchez-Meca J, Rosa-Alcázar AI, Marín-Martínez F, Gómez-Conesa A. Psychological treatment of panic disorder with or without agoraphobia: a meta-analysis. Clin Psychol Rev 2010 Feb;30(1):37-50. [CrossRef] [Medline]
- van Balkom AJ, Bakker A, Spinhoven P, Blaauw BM, Smeenk S, Ruesink B. A meta-analysis of the treatment of panic disorder with or without agoraphobia: a comparison of psychopharmacological, cognitive-behavioral, and combination treatments. J Nerv Ment Dis 1997 Aug;185(8):510-516. [CrossRef] [Medline]
- Andrews G, Issakidis C, Carter G. Shortfall in mental health service utilisation. Br J Psychiatry 2001 Nov 02;179(5):417-425. [CrossRef] [Medline]
- Coles ME, Coleman SL. Barriers to treatment seeking for anxiety disorders: initial data on the role of mental health literacy. Depress Anxiety 2010 Jan;27(1):63-71. [CrossRef] [Medline]
- Mojtabai R, Olfson M, Sampson NA, Jin R, Druss B, Wang PS, et al. Barriers to mental health treatment: results from the National Comorbidity Survey Replication. Psychol Med 2011 Aug;41(8):1751-1761 [FREE Full text] [CrossRef] [Medline]
- Allen AR, Newby JM, Mackenzie A, Smith J, Boulton M, Loughnan SA, et al. Internet cognitive-behavioural treatment for panic disorder: randomised controlled trial and evidence of effectiveness in primary care. B J Psych Open 2016 Mar;2(2):154-162 [FREE Full text] [CrossRef] [Medline]
- Berger T, Urech A, Krieger T, Stolz T, Schulz A, Vincent A, et al. Effects of a transdiagnostic unguided internet intervention ('velibra') for anxiety disorders in primary care: results of a randomized controlled trial. Psychol Med 2017 Jan;47(1):67-80. [CrossRef] [Medline]
- Ciuca AM, Berger T, Crişan LG, Miclea M. Internet-based treatment for panic disorder: a three-arm randomized controlled trial comparing guided (via real-time video sessions) with unguided self-help treatment and a waitlist control. PAXPD study results. J Anxiety Disord 2018 May;56:43-55. [CrossRef] [Medline]
- Andersson G. Guided internet treatment for anxiety disorders. As effective as face-to-face therapies? Stud Health Technol Inform 2012 Sep 2012:181-187. [CrossRef]
- Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry 2014 Oct;13(3):288-295 [FREE Full text] [CrossRef] [Medline]
- Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One 2010 Oct 13;5(10):e13196 [FREE Full text] [CrossRef] [Medline]
- Andrews G, Basu A, Cuijpers P, Craske M, McEvoy P, English C, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord 2018 Apr;55:70-78 [FREE Full text] [CrossRef] [Medline]
- Cuijpers P, Donker T, van Straten A, Li J, Andersson G. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med 2010 Dec;40(12):1943-1957. [CrossRef] [Medline]
- Haug T, Nordgreen T, Öst LG, Havik OE. Self-help treatment of anxiety disorders: a meta-analysis and meta-regression of effects and potential moderators. Clin Psychol Rev 2012 Jul;32(5):425-445. [CrossRef] [Medline]
- Mewton L, Smith J, Rossouw P, Andrews G. Current perspectives on internet delivered cognitive behavioral therapy for adults with anxiety and related disorders. Psychol Res and Beh Manage 2014 Jan:37. [CrossRef]
- Olthuis JV, Watt MC, Bailey K, Hayden JA, Stewart SH. Therapist-supported internet cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database Syst Rev 2015 Mar 05(3):CD011565. [CrossRef] [Medline]
- Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 2007 Mar;37(3):319-328. [CrossRef] [Medline]
- Hedman E, Ljótsson B, Rück C, Bergström J, Andersson G, Kaldo V, et al. Effectiveness of internet-based cognitive behaviour therapy for panic disorder in routine psychiatric care. Acta Psychiatr Scand 2013 Dec 14;128(6):457-467. [CrossRef] [Medline]
- ICT facts and figures. International Telecommunication Union. 2017. URL: https://www.itu.int/en/ITU-D/Statistics/Documents/facts/ICTFactsFigures2017.pdf [accessed 2021-02-08]
- Alpers GW. Ambulatory assessment in panic disorder and specific phobia. Psychol Assess 2009 Dec;21(4):476-485. [CrossRef] [Medline]
- Helbig S, Lang T, Swendsen J, Hoyer J, Wittchen H. Implementierung, akzeptanz und informationsgehalt eines ecological momentary assessment (EMA)-ansatzes bei patienten mit panikstörung und agoraphobie. Zeitschrift für Klinische Psychologie und Psychotherapie 2009 Apr;38(2):108-117. [CrossRef]
- Moskowitz DS, Russell JJ, Sadikaj G, Sutton R. Measuring people intensively. Cana Psychol 2009 Aug;50(3):131-140. [CrossRef]
- Myin-Germeys I, Oorschot M, Collip D, Lataster J, Delespaul P, van Os J. Experience sampling research in psychopathology: opening the black box of daily life. Psychol Med 2009 Feb 12;39(9):1533-1547. [CrossRef]
- Trull TJ, Ebner-Priemer UW. Using experience sampling methods/ecological momentary assessment (ESM/EMA) in clinical assessment and clinical research: introduction to the special section. Psychol Assess 2009 Dec;21(4):457-462 [FREE Full text] [CrossRef] [Medline]
- Heron KE, Smyth JM. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol 2010 Feb;15(Pt 1):1-39 [FREE Full text] [CrossRef] [Medline]
- Wang K, Varma DS, Prosperi M. A systematic review of the effectiveness of mobile apps for monitoring and management of mental health symptoms or disorders. J Psychiatr Res 2018 Dec;107:73-78. [CrossRef] [Medline]
- Van Singer M, Chatton A, Khazaal Y. Quality of smartphone apps related to panic disorder. Front Psychiatry 2015 Jul 14;6:96 [FREE Full text] [CrossRef] [Medline]
- Donker T, Petrie K, Proudfoot J, Clarke J, Birch M, Christensen H. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013 Nov 15;15(11):e247 [FREE Full text] [CrossRef] [Medline]
- Marcolino MS, Oliveira JAQ, D'Agostino M, Ribeiro AL, Alkmim MBM, Novillo-Ortiz D. The impact of mHealth interventions: systematic review of systematic reviews. JMIR Mhealth Uhealth 2018 Jan 17;6(1):e23 [FREE Full text] [CrossRef] [Medline]
- Sucala M, Cuijpers P, Muench F, Cardoș R, Soflau R, Dobrean A, et al. Anxiety: there is an app for that. A systematic review of anxiety apps. Depress Anxiety 2017 Jun;34(6):518-525. [CrossRef] [Medline]
- Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord 2017 Aug 15;218:15-22 [FREE Full text] [CrossRef] [Medline]
- Christoforou M, Sáez Fonseca JA, Tsakanikos E. Two novel cognitive behavioral therapy-based mobile apps for agoraphobia: randomized controlled trial. J Med Internet Res 2017 Nov 24;19(11):e398 [FREE Full text] [CrossRef] [Medline]
- Middleton KR, Anton SD, Perri MG. Long-term adherence to health behavior change. Am J Lifestyle Med 2013 Jun 14;7(6):395-404 [FREE Full text] [CrossRef] [Medline]
- Min JA, Lee CU, Lee C. Mental health promotion and illness prevention: a challenge for psychiatrists. Psychiatry Investig 2013 Dec;10(4):307-316 [FREE Full text] [CrossRef] [Medline]
- Proudfoot J, Clarke J, Birch M, Whitton AE, Parker G, Manicavasagar V, et al. Impact of a mobile phone and web program on symptom and functional outcomes for people with mild-to-moderate depression, anxiety and stress: a randomised controlled trial. BMC Psychiatry 2013 Nov 18;13:312 [FREE Full text] [CrossRef] [Medline]
- Ivanova E, Lindner P, Ly KH, Dahlin M, Vernmark K, Andersson G, et al. Guided and unguided acceptance and commitment therapy for social anxiety disorder and/or panic disorder provided via the internet and a smartphone application: a randomized controlled trial. J Anxiety Disord 2016 Dec;44:27-35. [CrossRef] [Medline]
- Bundesärztekammer. Beschlüsse des 114. Deutschen Ärztetages in Kiel, Stand. 2011. URL: https://www.bundesaerztekammer.de/fileadmin/user_upload/downloads/pdf-Ordner/MBO/MBO_Synopse.pdf [accessed 2011-08-29]
- Ebenfeld L, Stegemann SK, Lehr D, Ebert DD, Jazaieri H, van Ballegooijen W, et al. Efficacy of a hybrid online training for panic symptoms and agoraphobia: study protocol for a randomized controlled trial. Trials 2014 Nov 04;15(1):427 [FREE Full text] [CrossRef] [Medline]
- Bandelow B. Assessing the efficacy of treatments for panic disorder and agoraphobia. II. The panic and agoraphobia scale. Int Clin Psychopharmacol 1995 Jun;10(2):73-81. [CrossRef] [Medline]
- Bandelow B. Panic and Agoraphobia Scale (PAS). 1997. URL: https://www.testzentrale.de/shop/panik-und-agoraphobie-skala.html [accessed 2021-02-16]
- Schneider S, Margraf J. Agoraphobia and panic disorder. URL: https://www.hogrefe.com/de/shop/agoraphobie-und-panikstoerung-76509.html [accessed 2021-02-16]
- Beck AT, Steer RA, Brown GK. Beck depression inventory-second edition. BDI-II. URL: https://www.nctsn.org/measures/beck-depression-inventory-second-edition [accessed 2021-02-16]
- Hautzinger M, Keller F, Kühner C. Beck Depression Inventory Revision. BDI-II. 2006. URL: https://www.testzentrale.de/shop/beck-depressions-inventar.html [accessed 2021-02-16]
- Wittchen HU, Zaudig M, Fydrich T. SKID: structured clinical interview for DSM-IV; Axis I and II. In: Zeitschrift für Klinische Psychologie und Psychotherapie. Göttingen: Hogrefe; Jan 1999:68-70.
- Kleine Stegemann S, Ebenfeld L, Lehr D, Berking M, Funk B. Development of a mobile application for people with panic disorder as augmentation for an internet-based intervention. In: Proceedings of the Federated Conference on Computer Science and Information Systems. 2013 Presented at: 2013 Federated Conference on Computer Science and Information Systems; Sept 8-11, 2013; Krakow, Poland p. 1331-1337.
- Bandelow B, Broocks A, Pekrun G, George A, Meyer T, Pralle L, et al. Pharmacopsychiatry 2000 Sep;33(5):174-181. [CrossRef] [Medline]
- Bandelow B, Hajak G, Holzrichter S, Kunert HJ, Rüther E. Assessing the efficacy of treatments for panic disorder and agoraphobia. I. Methodological problems. Int Clin Psychopharmacol 1995 Jun;10(2):83-93. [CrossRef] [Medline]
- Wölk J, Sütterlin S, Koch S, Vögele C, Schulz SM. Enhanced cardiac perception predicts impaired performance in the Iowa Gambling Task in patients with panic disorder. Brain Behav 2014 Mar 22;4(2):238-246 [FREE Full text] [CrossRef] [Medline]
- Hamilton M. The assessment of anxiety states by rating. Br J Med Psychol 1959;32(1):50-55. [CrossRef] [Medline]
- Shear MK, Vander Bilt J, Rucci P, Endicott J, Lydiard B, Otto MW, et al. Reliability and validity of a structured interview guide for the Hamilton Anxiety Rating Scale (SIGH-A). Depress Anxiety 2001;13(4):166-178. [Medline]
- Weyer G. The Hamilton-Anxiety-Scale. In: Collegium Internationale Psychiatriae Scalarum; Internationale Skalen für Psychiatrie. 6th ed. Göttingen: Beltz-Test-GmbH; 2015.
- Chambless DL, Caputo GC, Bright P, Gallagher R. Assessment of fear of fear in agoraphobics: The Body Sensations Questionnaire and the Agoraphobic Cognitions Questionnaire. J Consult and Clin Psychol 1984 Dec;52(6):1090-1097. [CrossRef]
- Craske MG, Rachman SJ, Tallman K. Mobility, cognitions, and panic. J Psychopathol Behav Assess 1986 Sep;8(3):199-210. [CrossRef]
- Ehlers A, Margraf J, Chambless D. Body-related fears, cognitions, and avoidance questionnaires. AKV. URL: https://www.testzentrale.de/shop/fragebogen-zu-koerperbezogenen-aengsten-kognitionen-und-vermeidung.html [accessed 2021-02-16]
- Chambless DL, Caputo G, Jasin SE, Gracely EJ, Williams C. The mobility inventory for agoraphobia. Beh Res and Therapy 1985;23(1):35-44. [CrossRef]
- Hautzinger M, Bailer M. General depression scale. ADS. URL: https://www.testzentrale.de/shop/allgemeine-depressionsskala.html [accessed 2021-02-16]
- Radloff LS. The CES-D Scale. Applied Psychological Measurement 2016 Jul 26;1(3):385-401. [CrossRef]
- Crippa JAS, Osório FDL, Del-Ben CM, Filho AS, da Silva Freitas MC, Loureiro SR. Comparability between telephone and face-to-face structured clinical interview for DSM-IV in assessing social anxiety disorder. Perspect Psychiatr Care 2008 Oct;44(4):241-247. [CrossRef] [Medline]
- Irvine A, Drew P, Sainsbury R. ‘Am I not answering your questions properly?’ Clarification, adequacy and responsiveness in semi-structured telephone and face-to-face interviews. Qual Res 2012 Apr 05;13(1):87-106. [CrossRef]
- Rohde P, Lewinsohn PM, Seeley JR. Comparability of telephone and face-to-face interviews in assessing axis I and II disorders. Am J Psychiatry 1997 Nov;154(11):1593-1598. [CrossRef] [Medline]
- Lobbestael J, Leurgans M, Arntz A. Inter-rater reliability of the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID I) and Axis II Disorders (SCID II). Clin Psychol Psychother 2011 Jan 12;18(1):75-79. [CrossRef] [Medline]
- Gandek B, Ware JE, Aaronson NK, Apolone G, Bjorner JB, Brazier JE, et al. Cross-validation of item selection and scoring for the SF-12 Health Survey in nine countries. J Clin Epidemio 1998 Nov;51(11):1171-1178. [CrossRef]
- Ware J, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996 Mar;34(3):220-233. [CrossRef] [Medline]
- Boß L, Lehr D, Reis D, Vis C, Riper H, Berking M, et al. Reliability and validity of assessing user satisfaction with web-based health interventions. J Med Internet Res 2016 Aug 31;18(8):e234 [FREE Full text] [CrossRef] [Medline]
- Attkisson C, Zwick R. The client satisfaction questionnaire. Evaluation and Program Planning 1982 Jan;5(3):233-237. [CrossRef]
- Schmidt J, Lamprecht F, Wittmann WW. Satisfaction with inpatient management. Development of a questionnaire and initial validity studies. Psychother Psychosom Med Psychol 1989 Jul;39(7):248-255. [Medline]
- Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an Adjective Rating Scale. J Usability Stud. 2009 May. URL: http://uxpajournal.org/wp-content/uploads/sites/8/pdf/JUS_Bangor_May2009.pdf [accessed 2021-02-08]
- Brooke J. SUS: a 'Quick and Dirty' usability scale. London, UK: Taylor & Francis; 1996.
- Cohen J. Statistical power analysis for the behavioral sciences. 2nd edition. London, UK: Taylor & Francis; 1988.
- Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult ng and Clin Psychol 1991;59(1):12-19. [CrossRef]
- Schafer JL, Graham JW. Missing data: our view of the state of the art. Psychol Methods 2002 Jun;7(2):147-177. [Medline]
- Cuijpers P, Koole SL, van Dijke A, Roca M, Li J, Reynolds CF. Psychotherapy for subclinical depression: meta-analysis. Br J Psychiatry 2014 Oct 02;205(4):268-274 [FREE Full text] [CrossRef] [Medline]
- Marks IM, Kenwright M, McDonough M, Whittaker M, Mataix-Cols D. Saving clinicians' time by delegating routine aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol Med 2004 Jan 14;34(1):9-17. [CrossRef] [Medline]
- Bruinsma A, Kampman M, Exterkate CC, Hendriks GJ. An exploratory study of 'blended' cognitive behavioural therapy (CBT) for patients with a panic disorder: results and patients' experiences. Tijdschr Psychiatr 2016;58(5):361-370 [FREE Full text] [Medline]
- mHealth: new horizons for health through mobile technologies. World Health Organization. 2011. URL: https://www.who.int/goe/publications/goe_mhealth_web.pdf [accessed 2021-02-08]
- Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2005 Jun;62(6):617-627 [FREE Full text] [CrossRef] [Medline]
- Roy-Byrne PP, Stang P, Wittchen H, Ustun B, Walters EE, Kessler RC. Lifetime panic-depression comorbidity in the National Comorbidity Survey. Association with symptoms, impairment, course and help-seeking. Br J Psychiatry 2000 Mar 02;176(3):229-235. [CrossRef] [Medline]
- Walderhaug EP, Gjestad R, Egeland J, Havik OE, Nordgreen T. Relationships between depressive symptoms and panic disorder symptoms during guided internet-delivered cognitive behavior therapy for panic disorder. Nord J Psychiatry 2019 Oct 02;73(7):417-424. [CrossRef] [Medline]
- Allen LB, White KS, Barlow DH, Shear MK, Gorman JM, Woods SW. Cognitive-Behavior Therapy (CBT) for panic disorder: relationship of anxiety and depression comorbidity with treatment outcome. J Psychopathol Behav Assess 2010 Jun 24;32(2):185-192 [FREE Full text] [CrossRef] [Medline]
- Ebert DD, Van Daele T, Nordgreen T, Karekla M, Compare A, Zarbo C, et al. Internet- and mobile-based psychological interventions: applications, efficacy, and potential for improving mental health. European Psychologist 2018 May;23(2):167-187. [CrossRef]
- Bower P, Gilbody S. Stepped care in psychological therapies: access, effectiveness and efficiency. Narrative literature review. Br J Psychiatry 2005 Jan;186:11-17. [CrossRef] [Medline]
|3M-FU: 3-month follow-up|
|6M-FU: 6-month follow-up|
|ACQ: Agoraphobic Cognitions Questionnaire|
|ANCOVA: analysis of covariance|
|BDI-II: Beck Depression Inventory II|
|BSQ: Body Sensations Questionnaire|
|CBT: cognitive behavioral therapy|
|CES-D: Center for Epidemiologic Studies Depression Scale|
|CSQ-I: Client Satisfaction Questionnaire adapted to internet-based interventions|
|HAM-A: Hamilton Anxiety Scale|
|iCBT: internet-based interventions based on cognitive behavioral therapy principles|
|IG: intervention group|
|MI: mobility inventory|
|NNT: numbers needed to treat|
|PAS: Panic and Agoraphobia Scale|
|RCI: Reliable Change Index|
|SCID-I: Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition Axis I Disorders|
|SF-12: 12-Item Short-Form Health Survey|
|SUS: System Usability Scale|
|WLC: wait-list control|
Edited by G Eysenbach; submitted 29.05.20; peer-reviewed by M Feijt, K Mathiasen; comments to author 29.06.20; revised version received 06.10.20; accepted 17.01.21; published 04.03.21Copyright
©Lara Ebenfeld, Dirk Lehr, David Daniel Ebert, Stefan Kleine Stegemann, Heleen Riper, Burkhardt Funk, Matthias Berking. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 04.03.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.