To evaluate the efficacy of a hybrid web-based training program for panic disorder (with and without agoraphobia), we conducted a prospective, two-arm randomized trial, in which 92 participants with significant symptoms of panic disorder were randomly allocated either to the GET.ON Panic intervention group (IG) who received the training program immediately or to the WLC group who received the training program 6 months after randomization. For randomization, we used the automated computer program DatInf RandList version 1.2 (DatInf GmbH). The allocation was stratified for clinical or subclinical symptomatology as well as the presence or absence of agoraphobia in the order of incoming informed consent. To include equal numbers of participants in each group, we used block randomization (n=2 per block). The staff conducting the diagnostic interviews and observer ratings were blinded to the participants’ randomization statuses. The participants could participate in the training program with a pseudonym of their own choice. Ethical approval for this study was obtained from the Ethical Committee of the University of Marburg (registration number: 2013-23 K). The study was preregistered with the German Clinical Trial Register (registration number: DRKS00005223). The study protocol was submitted for publication before randomization [45].

The study participants were recruited from the general population between August 2013 and October 2014. Announcements in newspapers, support groups, and social media, such as Facebook, guided interested individuals to the web-based health center website of our research group, where they could apply on the web to participate in the study. Applicants were asked to complete a web-based questionnaire assessing the following inclusion criteria: (1) experiencing mild-to-moderate panic symptoms as assessed by the PAS (score range: 9-28) [46,47], (2) being aged ≥18 years, (3) having panic as the primary concern for seeking help, (4) having internet and smartphone access with minimum system requirements of iPhone (TM) 3GS (Apple Inc; iOS 6 and iOS 7) or a comparable Android device (Android 2.3 or newer), and (5) providing their informed consent.

The exclusion criteria were as follows: (1) receiving current psychological help for anxiety problems or being on a wait-list for psychotherapy; (2) having physical health problems that were assessed via a self-report that prevents participants from engaging in self-exposure, as recommended by the German guideline for treating people with panic disorder and agoraphobia [48]; (3) currently having posttraumatic stress disorder or psychotic or dissociative disorders assessed via self-report and clinical interview; and (4) having current suicidality, as assessed by a score above 1 on item 9 of the Beck Depression Inventory II (BDI-II) [49,50] and question A9 of the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition Axis I Disorders (SCID-I) [51]. In the event that potential participants were excluded because of suicidal ideation or intention, they were given information about further help according to an established suicide protocol. All excluded participants were contacted via email and provided with information regarding where they could obtain appropriate help.

Participants in the treatment condition received the GET.ON Panic treatment, which is a hybrid (ie, desktop-based and smartphone-based), iCBT-based self-help intervention for treating symptoms of panic disorder [45]. Participants were advised to log on to the training platform, which was provided by the technical partner Minddistrict GmbH on a weekly basis and consecutively work through the following sessions: (1) psychoeducation, (2) interoceptive exposure, (3) in vivo exposure, (4) cognitive restructuring—introduction, (5) cognitive restructuring—extension, and (6) relapse prevention. In addition, participants were instructed on the complementary use of the GET.ON Panic app [52]. The app supported participants in (1) completing their homework assignments (eg, keeping an anxiety diary); (2) planning, conducting, and evaluating interoceptive and in vivo exposure tasks; and (3) performing relaxation exercises (Table 1).

After every session, participants received written feedback from a trained coach based on a coaching manual developed by members of our research group to ensure a standardized procedure of coaching (the manual is available on request). The guidance focused on increasing motivation and adherence throughout the training progress, rather than providing individual therapeutic advice. The average feedback took about 20-30 min. Coaches also sent reminders via a secure messaging system within the training platform if participants did not log on for 1 week. All coaches had a degree in psychology and were supervised by a licensed clinical psychologist.

The mobile app contains a diary for recording and monitoring panic-related symptoms, such as panic events, degree of avoidance behavior, general anxiety, and mood level on a visual analog scale (0-10). Furthermore, the app recorded the type and number of exposure exercises performed by the participant. In addition, we used the System Usability Scale (SUS) at postassessment (T2) to assess the usability of the GET.ON Panic app [74,75]. The sum score ranges from 0 to 100, with a higher score indicating better usability.

Participants completed a sociodemographic questionnaire, the PAS and the Suicidality item of BDI-II at screening (T0); at baseline (T1), we assessed the PAS, the SCID-I, the HAM-A, the ACQ, the BSQ, the MI, the CES-D, and the SF-12; at postassessment (T2), we assessed the PAS, the HAM-A, the ACQ, the BSQ, the MI, the CES-D, the SF-12, the CSQ-I (only IG), and the SUS (only IG); at 3-month follow-up (3M-FU; T3), we assessed the PAS, the ACQ, the BSQ, the MI, the CES-D, and the SF-12; and at 6M-FU (T4), we assessed the PAS, the SCID-I, the HAM-A, the ACQ, the BSQ, the MI, the CES-D, and the SF-12. Diary data were continuously measured during the training period and beyond (Table 2).

To assess treatment efficacy, the GET.ON Panic group was compared with the WLC group on all outcome measures (T2, T3, and T4) using univariate analyses of covariance (ANCOVAs) with the baseline scores as covariates. On the basis of a previous meta-analysis [21], we powered the study to detect an effect size of Cohen d=0.6 (1− of 80%; =.05) with intention-to-treat (ITT) at T2 as our primary level of analysis. Accordingly, a sample size of 90 was required. Cohen d [76] was used to measure effect size. To account for covariance, we calculated Cohen d over the partial eta squared (η²). To assess a clinically reliable change of panic severity (response) on an individual level, we calculated the Reliable Change Index (RCI) as proposed by Jacobson and Truax [77], coded participants as responders or deteriorators if their score on the PAS differed by 10.68 points on the PAS, and performed a Pearson chi-square test to compare the reliable change of the GET.ON Panic group with the WLC. Corresponding to the RCI, we calculated numbers needed to treat (NNT) score indicating how many participants must take part in order for GET.ON Panic to achieve one clinically relevant improvement. To assess remission rates, we calculated the percentage of people who had a diagnostic status of panic disorder according to the SCID-I interview at baseline (T0) and at the 6M-FU (T4) and performed a Pearson chi-square test to compare the diagnostic status of the GET.ON Panic group with the WLC covering a period of the last 3 months.

Missing data at postassessment, 3M-FU assessment, and 6M-FU assessment were performed using a Markov Chain Monte Carlo multivariate imputation algorithm (SPSS 23) with hundred estimations per missing value and all available data on outcomes, age, and gender as predictors [78].

Over a period of 14 months, a total of 235 individuals completed the screening questionnaire. Of these, 117 did not meet the inclusion criteria or matched one or more exclusion criteria (Figure 1). Severe panic symptom severity (n=54), current psychotherapy (n=34), or physical contraindications (n=23) were the most frequent reasons for exclusion. The remaining 118 candidates were eligible to participate in the clinical interviews. Of those, 19 did not provide informed consent. After this interview, another 6 candidates were excluded because they did not have panic symptoms as their primary reason for seeking help. All excluded individuals were provided with information about applicable health care system services. The remaining 92 participants were randomly assigned to the hybrid web-based training program GET.ON Panic or the WLC condition.

Most participants were female (51/92, 55%), White (76/92, 83%), on average aged 38 years (SD 10.42), highly educated (60/92, 65%), married or in a relationship (82/92, 89%), and currently working (51/92, 55%). On the basis of the SCID interview, the most common diagnosis was panic disorder with agoraphobia (78/92, 83%). A significant number of patients (12/92, 13%) met the criteria for panic disorder without agoraphobia. Of all participants, 26% (24/92) met the criteria for at least one additional anxiety disorder, in addition to panic disorder. A small percentage (2/92, 2%) had a current major depressive episode as a comorbid condition. Most participants (58/92, 63%) reported that they had previously undergone psychotherapeutic treatment (Multimedia Appendix 1).

Baseline data were available for all the participants. The attrition rate was 9% (8/92) at postassessment (4/45 in the IG and 4/47 in the WLC), 18% (17/92) at the 3M-FU (10/45 in the IG and 7/47 in the WLC), and 22% (20/92) at the 6M-FU (10/45 in the IG and 10/47 in the WLC; Figure 1).

On average, the number of completed sessions in the GET.ON Panic group was 5.11 (SD 1.67). All 6 sessions were completed by 73% (33/45) of the participants, 4% (2/45) of the participants completed only session 1, 11% (5/45) of the participants dropped out after session 2, 7% (3/45) of the participants were lost after session 3, 2% (1/45) of the participants stopped the training after completing session 4, and 2% (1/45) of the participants after session 5. In total, 27% (12/45) of the participants did not complete the training. The reasons for intervention dropout were mentioned for 33% (4/12) of them: lack of time, lack of motivation, lack of personal contact with the eCoach, or surgery that interfered with completing the intervention. The resting 67% (8/12) of the participants were also study dropouts, and no reasons for stopping the intervention were known because they did not complete the postassessment.

Preliminary analyses indicated that all necessary conditions for the intended statistical analyses were met. There was a greater decrease in self-reported panic disorder symptom severity in the intervention condition than in the WLC condition (Multimedia Appendices 2 and 3). With regard to the primary outcome, participants in the GET.ON Panic condition reported significantly lower (baseline-controlled) panic symptom severity at posttreatment than the WLC group (F=9.77; P=.002; partial η²=0.10; Cohen d=0.66; 95% CI 0.24-1.08). This effect became even stronger (F=17.40; P<.001; partial η²=0.16; Cohen d=0.89; 95% CI 0.46-1.31) at the 3M-FU and remained significant (F=14.63; P<.001; partial η²=0.14; Cohen d=0.81; 95% CI 0.38-1.24) at the long-term 6M-FU. The effect sizes ranged from medium to large.

With regard to observer-based ratings, ANCOVA showed a significant difference in anxiety symptoms between groups as measured by the HAM-A at postmeasurement (F=3.97; P=.049; partial η²=0.04; Cohen d=0.42; 95% CI 0.01-0.84) and at the 6M-FU (F=4.86; P=.03; partial η²=0.05; Cohen d=0.47; 95% CI 0.05-0.88) with small-to-medium effect sizes. Further analyses indicated that the findings did not significantly change when the analyses were based on the study completer instead of the ITT sample.

With regard to response, the reliable clinical changes were not significant at postmeasurement (χ²₂ [n=92]=2.5; P=.28; improvement: GET.ON Panic: 12/45, 27% and WLC: 7/47, 15%; deterioration: GET.ON Panic: 2/45, 4% and WLC: 1/47, 2%) or at the 3M-FU (χ²₂ [n=92]=5.3; P=.07; improvement: GET.ON Panic: 14/45, 31% and WLC: 6/47, 13%; deterioration: GET.ON Panic: 0/45, 0% and WLC: 1/47, 2%). However, the GET.ON Panic group was superior to the WLC in terms of the percentage of participants attaining reliable clinical change (RCI=±10.68) in panic symptom severity at the 6M-FU (χ²₂ [n=92]=6.0; P=.049; improvement: GET.ON Panic: 22/45, 49% and WLC: 12/47, 26%; deterioration: GET.ON Panic: 0/45, 0% and WLC: 1/47, 2%). These reliable clinical changes correspond to NNT from baseline to posttreatment of 8.49 (95% CI 3.54 to >10⁶), at the 3M-FU of NNT=5.45 (95% CI 2.87-55.78), and at the 6M-FU of NNT=4.28 (95% CI 2.35-24.07). Regarding the long-term effect, these results indicate that 4 individuals had to participate in the GET.ON Panic training program to result in one additional individual having reliable clinical improvement in panic symptom severity.

With regard to remission rates, at baseline, nearly all participants (90/92, 98%) fulfilled the diagnostic criteria for panic disorder. At the 6M-FU, 76% of the participants (70/92, 76% GET.ON Panic group: 33/45, 73%; WLC: 37/47, 79%) agreed to the telephone-administered diagnostic interview. In total, 21% (15/70) of the participants were free of a diagnosis. There was a greater reduction in diagnoses in the GET.ON Panic group (11/33, 33%) than in the WLC group (4/37, 11%; χ²₁[n=70]=5.3; P=.02).

Comparing the GET.ON Panic with the WLC group on further self-rated anxiety measurements, we found stronger between-group effect sizes for agoraphobic cognitions (partial η²=0.06; Cohen d=0.51; 95% CI 0.05-0.93) and bodily sensations (partial η²=0.05; Cohen d=0.46; 95% CI 0.05-0.88) in the GET.ON Panic group than in the WLC group at posttreatment. With regard to follow-up measurements, these effects remained stable for both agoraphobic cognitions (partial η²=0.07; Cohen d=0.55; 95% CI 0.14-0.97 after 3 months and partial η²=0.05; Cohen d=0.46; 95% CI 0.04-0.87 after 6 months) and bodily sensations (partial η²=0.14; Cohen d=0.79; 95% CI 0.37-1.22 after 3 months and partial η²=0.09; Cohen d=0.66; 95% CI 0.22-1.06 after 6 months). A difference in agoraphobic avoidance between the groups could only be found when participants had to manage difficult situations when they were alone with small effect sizes (partial η²=0.05; Cohen d=0.45; 95% CI 0.04-0.86) at posttreatment and a medium effect size (partial η²=0.11; Cohen d=0.70; 95% CI 0.27-1.12) at the 6M-FU. ANCOVA did not reveal a significant difference between the groups regarding agoraphobic avoidance when participants had to manage difficult situations when they were in companionship with other people (Multimedia Appendices 2 and 3).

At the postmeasurement as well as at the 3M-FU, the GET.ON Panic group showed no significant reduction in depressive symptoms compared with the WLC group (Multimedia Appendices 2 and 3). However, at the 6M-FU, the depressive symptoms of the GET.ON Panic group decreased significantly with a small effect size compared with the WLC group (partial η²=0.06; Cohen d=0.49; 95% CI 0.07-0.90). The results on the quality of life scales with regard to mental health showed no reduction at postmeasurement or at the 3M-FU but a medium reduction after 6 months (partial η²=0.11; Cohen d=0.70; 95% CI 0.28-1.12). Furthermore, no differences in symptoms regarding physical aspects of quality of life were found.

The participants of the training group (n=45) used the mobile diary on average 25.02 times (SD 19.48; range 0-56) during the 8-week training period on average (0.45 diary entries per day per participant). The repeated analysis of variance did not reveal any changes in the diary scores over a period of 8 weeks. Furthermore, they were not related to the primary outcome. The participants performed an average of 149.80 (SD 279.34; range 0-1702) interoceptive exposure exercises and 6.63 in vivo exercises (SD 17.74; range 0-113). The mean SUS score was 71.16 (SD 18.97) at posttreatment, which indicates good usability of the GET.ON Panic app. Overall, user satisfaction with the hybrid training program was high (mean 28.10, SD 5.09). For example, 91% of the participants indicated that they would recommend the training program to a friend in need.

This study aims to evaluate the efficacy of GET.ON Panic, a guided, mobile- and web-based CBT training program for adults with significant panic disorder symptoms. The results show that individuals treated with GET.ON Panic experienced a significantly greater reduction in panic disorder symptom severity than did participants in a WLC condition with between-group effect sizes of Cohen d. The findings also show that the effects were not only stable over time but even increased after the treatment was completed (Cohen d/NNT=0.66/8.49 at posttreatment vs Cohen d/NNT=0.89/5.45 after 3 months and Cohen d/NNT=0.81/4.28 after 6 months).

As such, they fall well into the range of reported effect sizes in meta-analyses for internet-based interventions for panic disorder (eg, Hedge g=0.83 [18]; g=1.31 [19]; Hedge g=0.83 [21]; Cohen d=0.96 [24]). The findings also showed that one of 3 participants in the IG had attained complete remission of panic disorder at the last assessment point, whereas this was only the case in one of 10 participants in the control group. With regard to secondary outcomes, it is of note that the 6M-FU effects on depressive symptoms are larger than the average of effects reported for psychological treatment for depression (Cohen d=0.49 vs Hedge g=0.35 [79]). Finally, in this study, adherence rates and user satisfaction were slightly higher than those reported in previous studies (adherence: 73% vs 66%; satisfaction: 91% vs 86% [19]).

A potential step-up could be the use of an intervention that integrates hybrid web-based training program into f2f CBT [80]. In such blended interventions, therapists might fully exploit the potential of using the ecological momentary assessment data provided by the smartphone as well as the potential of ecological momentary interventions derived from individual case formulations and carried into the patient’s life with the help of their mobile devices [81]. The adherence and usability rates in this study appear to be superior to what is currently reported for desktop-based iCBT interventions. This suggests that the integration of mobile components into iCBT should be a focus of future studies. The rapidly shifting use of mobile- instead of desktop-based devices underlines this [26,82].

The finding that depressive symptom severity was significantly reduced in the IG is important, as many individuals with panic disorder also have other mental health problems such as depression [83]. As cooccurring disorders may mutually help maintain each other [5,84], it is important that comorbid conditions are treated along with the primary disorder. The positive effects of the hybrid intervention evaluated in this study on depression are consistent with the findings from CBT that successfully treat panic disorder, which also result in a reduction of depressive symptoms [85,86].

To our knowledge, this is the first study to examine the efficacy of iCBT training program that makes use of mobile components in a target group of people with mild-to-moderate panic and agoraphobia symptoms. One of the main strengths of this study is its solid study design, which tests a newly developed training program within a randomized controlled trial against a WLC. In addition to self-rating outcomes, we conducted clinical interviews with regard to symptom severity and changes in diagnostic status over a period of 6 months and an observer-rated anxiety outcome to validate the outcomes based on self-ratings. Furthermore, this study has high ecological validity, as participants used their own smartphones. They were supposed to interact with their smartphones as they would normally do. This may lead to a higher acceptance of and satisfaction with the GET.ON Panic training program and foster the integration of psychological interventions into the daily lives of individuals. The overall low dropout rates in this study support this assumption.

This study has several limitations that need to be considered. First, the study results cannot be generalized to all individuals with panic disorder symptoms. Participants who took part in this trial actively participated and underwent an extended eligibility procedure before the study. Many interested individuals were excluded based on the criteria defined in the study protocol. Thus, we assume that the current participants represent a more intrinsically motivated study sample and, in addition, have a higher affinity for the internet than the average individuals with panic disorder. Therefore, the external validity of this study might be limited. Second, for future treatment development, it would have been of interest to compare the hybrid intervention with both an exclusively desktop-based and an exclusively mobile-based intervention for panic disorder. However, as such a design was beyond what we could realize in this study, it would need to be used in subsequent studies. Such studies should also compare the efficacy (and cost-effectiveness) of desktop-, mobile-based, hybrid, and blended interventions with f2f therapy for panic disorder. Third, we cannot draw any conclusions on the efficacy beyond the 6M-FU assessment. Thus, future studies should evaluate the long-term effects of hybrid iCBT interventions for panic disorder.

The results of this study suggest that a significant number of individuals with symptoms of panic disorder can be helped with an intervention that is comparatively easy to disseminate and that can be used anonymously, which arguably lowers an important barrier to service utilization [87]. However, the results also show that about two-thirds of the participants had not completely recovered after the intervention. Thus, interventions such as GET.ON Panic might best be used in a step-by-step care framework in which patients failing to attain recovery through an internet-based intervention subsequently receive more intense (and costly) interventions [88].