The study was a Web-based, assessor-blinded, parallel-group RCT with an active control group. Participants were individually randomly allocated (ratio of 1:1) to either the treatment group or the control group at baseline and were given an equal amount of time to complete each intervention (12 weeks). Data were gathered automatically through online collection of anonymized data, without any researcher intervention. The primary outcome was the degree of symptom severity, as measured by the self-reported version of the Panic and Agoraphobia Scale [27] at end point.

A website was set up for the trial on which advertisements and all relevant information and updates were posted. The website was hosted by HeLP Ltd (currently known as Thrive Therapeutic Software). Advertisements of the trial were also posted on social media (eg, Facebook, Twitter), support groups and forums, websites of relevant organizations such as Anxiety UK, blogs, and university websites. We also individually contacted members of anxiety support groups via Facebook or forum messages. We created a mailing list of people subscribing their interest in the trial, whom we encouraged to stay in touch until the trial commenced. Moreover, contacts of the Chief Investigator that were working in health-related settings were encouraged to inform any relevant clients or representatives about the study. There were 2 rounds of recruitment in order to achieve a larger number of participants.

We always contacted participants via email. Initially, those who expressed interest in participating followed a link to the online information sheet and consent form, which outlined the eligibility criteria and information about the trial. Through consenting to participate and answering a series of questions, participants confirmed that they met the criteria and understood the purpose of the study. Participants also provided their email addresses and names, though the latter was optional. Participant codes were then assigned to those who consented, and details about their mobile devices (smartphones or tablets) and demographics (age and sex) were obtained. Participants were instructed to use only their participant code to identify themselves.

Participants received invitations for the apps, which were available to use for free. Emails also contained links to weekly surveys on app use and links to questionnaires, a description of the specific survey questionnaire, links to the calendar and the main website of the trial, information about the upcoming survey, and useful contact details. A Frequently Asked Questions section was set up on the main website. Data were collected online and could only be accessed by the researchers.

The primary outcome was the severity of agoraphobic and panic symptoms, measured by the PAS [27]. The questionnaire was administered online in a self-report format at baseline, midpoint (6 weeks), and end point (12 weeks) of the trial. Reminders were emailed to those who did not reply to the questionnaires before the prespecified deadline. Participants rated the symptoms they experienced in the previous week on a 5-point scale. The questionnaire comprises 14 items, although only 13 of those are used to calculate severity scores. The items are grouped into 5 subscales: (1) Panic Attacks, assessing frequency, severity, and duration of panic attacks, (2) Agoraphobic Avoidance, assessing frequency of avoidance, and number and relevance of avoided situations, (3) Anticipatory Anxiety, assessing frequency and severity of anxiety, (4) Disability, assessing impairment in family life, social relationships, and employment, and (5) Worries about Health, assessing worries about damage to health and assumption of organic disease. The scale was originally validated in a sample of 235 panic patients and has shown good internal consistency (Cronbach alpha=.88), test-retest reliability, and good internal and external validity [27,30]. Cronbach alphas for this study’s sample indicated good internal consistency, with alpha=.84 for the overall scale. Coefficients for individual subscales ranged from .41 to .88. The scale has been shown to be sensitive to change due to treatment in 2 clinical trials [31,32].

The secondary outcomes were completion of the intervention and engagement with the apps. Completion of the interventions was assessed in the short online surveys that were sent weekly, by asking participants if they had used the app, how much time they used it for over the past week, and how many sessions they had completed. If participants claimed that they had not used the app, they were asked to give reasons.

We estimated the sample size on the basis of using the self-rated version of the PAS (SD 10.3) as the primary outcome measure. At least 68 participants in each arm were needed to detect a 5-point between-group difference, with a 2-sided significance level set at 5% and power at 80%. Given the high dropout rates in Web-based trials [33], the aim was to recruit at least 150 participants in total.

We used a random computer-generated sequence to randomly allocate participants to the 2 intervention groups. The random allocation sequence was retrieved from a website that generates truly random numbers [34], by a person outside the research team. We applied block randomization to ensure equal numbers of participants in each group (ratio of 1:1). Participants were automatically allocated to intervention groups by a formula on Excel version 14.6.6 (Microsoft Corporation) using the random number sequence, which was coordinated by another contact who was not a member of the research team. The random sequence and the allocation of participants to groups were concealed from research staff throughout the trial. The same person sent emails to participants containing the link to the assigned app after they had returned the baseline questionnaire.

The trial was assessor blinded, as researchers were blinded to treatment allocation throughout the trial and during the statistical analysis. This was achieved by having a person outside the research team to manage treatment allocation and personal communications with the participants. Any questions or comments made in the surveys were forwarded from this contact to research staff, excluding any participant details or codes. This was to ensure that the researchers remained blinded to treatment allocation, as some comments contained information about the specific app the participants were using. Researchers did not have any access to data regarding treatment allocation, as those were stored on a secure database, separate and protected from other research files. During data collection and analysis, only numerical codes were used to indicate group allocation. Researchers did not know which groups those referred to until the end of the analysis.

Blinding the participants was not possible, as the apps were clearly labelled as “Stress Free” and “Agoraphobia Free,” and masking those would require significant changes in the software, which were not feasible. Moreover, even if the apps were not differentially labelled, it is very likely that participants would have become aware of which treatment group they were assigned to because of the intervention content and the extent to which it addressed agoraphobia.

The research was approved by the Roehampton University Ethics Committee (reference number: PSYC 14/ 117). The trial was registered and reported in accordance with the Consolidated Standards of Reporting Trials (CONSORT)-EHEALTH checklist (Multimedia Appendix 3) [35]. Data were kept anonymized and protected according to the UK Data Protection Act [36]. We could obtain information about adverse events or effects of the interventions from feedback participants provided in the weekly surveys.

We analyzed the data on Stata version 14 (StataCorp LLC). We checked baseline data for normality and obtained descriptive statistics to capture the demographic and clinical characteristics of the sample. The analysis performed was intention-to-treat, as requested in the CONSORT [37]. All participants who completed the baseline assessment were included in the analysis as they were randomly allocated.

We used a linear mixed model to analyze the data, with a random effect of participant, and fixed effects of time (baseline, midpoint, end point), group (Agoraphobia Free and Stress Free), and the interaction between time and group. The estimated baseline PAS score was constrained to be identical in the 2 groups, thus adjusting for baseline and allowing the relationship between baseline and follow-up scores to differ at each time point. Another advantage of this statistical method is that the data from all participants contribute to the analysis, even if there is a substantial amount of missing data at follow-up [38]. We used an unstructured residual covariance matrix to allow for correlations within participants between the different time points. Statistical significance was taken at the 5% level (P<.05).

We conducted a planned secondary completers’ analysis using the same data analytic strategy as the intention-to-treat analysis. This analysis included only those participants who were identified as intervention completers.

The first phase of recruitment started from September 2014 and ended in late February 2015, and the second phase started in March 2015 and concluded in April, 2015. In the first round, 153 participants consented to participate, and we recruited 17 additional participants in the second round. The procedure following recruitment was the same in both samples. Data collection ended in June 2015. Figure 1 shows how the total sample of 170 participants progressed through the trial.

After 171 individuals consented to participate, 1 wanted to withdraw from the trial. The rest were randomly allocated into the 2 arms, but were not told which app they were assigned to at that point. Although the aim was to assign equal numbers of participants to each group, at each stage of randomization, the treatment group happened to have 1 more person than the other group, because the number of participants recruited each time was odd, and researchers were blinded to the allocation sequence.

We asked participants to fill in a form with their demographic and device details to register for the intervention so that we could set up the corresponding app invitations. A total of 18 participants did not return the form and did not reply to emails, even after they had been sent several reminders; thus, we treated them as dropouts and excluded them from the trial. The baseline PAS questionnaire was then sent and had to be completed in order for participants to proceed and receive the intervention. A total of 10 individuals did not return the questionnaire and did not reply to any emails. Up to this point, participants were still unaware of the group they were assigned to; thus, their exclusion from the trial was very unlikely to introduce any bias. The remaining 142 who completed the PAS were sent their allocated intervention, but 6 participants could not download the app on their device, despite efforts to resolve the technical issues.

After baseline, 39 participants did not want to continue with the trial and dropped out, while 29 participants did not reply to emails and did not complete the questionnaires. We included all participants who provided baseline data in the analysis.

Of the 142 participants who completed the baseline assessment, 2 did not give details of their age and sex (1 from each treatment arm). Of the remaining 140, 118 (84.3%) were female and had a mean age of 39.7 years (SD 11.3). Overall, the mean participant PAS total score was 30.3 (SD 8.7), and the scores were normally distributed (Shapiro-Wilk test: W=.985, P=.14).

The mean PAS total score fells into the severe range and was much higher than that of the original sample used in the validation of the scale [27], which reported a mean score of 23.5, SD 10.3. Our sample also had a higher proportion of women than the original sample (of whom only 57% were women) and a slightly higher age (mean 36.09 years in the original sample). A significant proportion of our sample (114/142, 80.3%) had severe or extremely severe Agoraphobic Avoidance, while 85.2% (121/142) avoided more than 8 situations. None of the participants scored 0 on the Agoraphobic Avoidance subscale, which indicates that all of them experienced agoraphobic symptoms to some degree. The lowest score on that scale was 1.67, indicating mild Agoraphobic Avoidance. In contrast, 29 participants (20.4%) obtained a score of 0 on the Panic Attacks subscale, which suggests that some participants experienced agoraphobia in the absence of panic attacks. Overall, agoraphobic symptoms were more prominent than panic symptoms in this sample.

Table 2 shows the demographic and clinical baseline characteristics of the participants by group. The 2 groups did not differ statistically on any of those characteristics at baseline (all P>.05).

A total of 68 (47.9%) participants had missing outcomes. Overall, the differences between participants with missing data and those without were not statistically significant on any of the baseline variables examined. There were no significant differences in age (t₁₃₈=0.85, P=.40) sex (χ²₁=0.6, P=.45), or clinical symptom severity (t₁₄₀=1.32, P=.19). There were 37 participants (54%) with missing data in the Agoraphobia Free arm and 31 (46%) in the Stress Free arm. The relationship between missing data and treatment arm was not significant (χ²₁=0.5, P=.49). Therefore, participant attrition did not seem to be biased with regard to group or any other baseline factor.

We produced a linear mixed model assessing the relative effects of each intervention on PAS scores at the 2 follow-up time points. Table 3 presents the estimated differences in PAS scores for the Agoraphobia Free group compared with the Stress Free group adjusted for baseline score at the 2 time points.

At end point, symptom severity scores decreased in both groups, but there was no evidence that the changes were significantly greater among participants of the Agoraphobia Free group than among those in the Stress Free group. Similarly, at midpoint there were no significant differences in symptom severity changes between the 2 groups. Therefore, there were no significant differences between the 2 groups. Figure 2 presents the differences on the primary outcome for each group over time.

We carried out the same linear mixed model analysis (n=142) using each PAS subscale as the dependent variable to examine whether there was a difference between the 2 groups in terms of symptom dimensions. We found no significant interactions between group and time for any of those outcomes (all P>.05).

We conducted within-group contrasts to examine the degree of change in symptom severity over time. For both Agoraphobia Free (n=73) and Stress Free (n=69), there were statistically significant improvements in symptom severity from baseline to midpoint and end point. Table 4 presents the results.

Data on completion were available from participants who consistently replied to the surveys and questionnaires (n=74). There were 56 participants who completed 80% or more of the assigned intervention; 25 of those received Agoraphobia Free and 31 received Stress Free. Of those who were deemed noncompleters, 11 were in the Agoraphobia Free arm and 7 in the Stress Free arm. There was no relationship between treatment arm and completion of intervention (χ²₁=1.5, P=.22).

We examined differences in completers’ symptom severity between the 2 intervention groups. We produced a linear mixed model, with random effect of participant, and fixed effects of time (baseline, midpoint, end point), group (Agoraphobia Free and Stress Free), and the interaction between time and group. In line with the intention-to-treat analysis, there were no significant differences between the 2 groups at end point or midpoint, as Table 5 shows. The within-group changes at each follow-up time point compared with baseline were significant in both groups (all P<.001). Therefore, there were significant reductions in symptom severity in both groups over time, but those reductions were equivalent across the 2 groups.

Wichmann et al [39] recently showed that an overall postintervention (ie, face-to-face CBT) decrease in the total PAS score of about 4 to 5 points represents a clear clinical change in terms of quality of life and functioning. This is slightly smaller than the change we observed in our study (6 points) for end point PAS scores compared with baseline. The decrease in symptom severity was even more marked in the completers’ sample (7 points in the Stress Free group and 10 points in the Agoraphobia Free group). Therefore, it seems that both apps were overall successful in achieving clinically significant change.

Of the 74 participants who provided data at end point, 46 (62%) had a reduction of 5 or more points on the PAS at end point. A total of 25 (66%) participants in the Stress Free group and 21 (58%) participants in the Agoraphobia Free group improved at least 5 points on the PAS at end point.

A total of 7 participants (Agoraphobia Free: n=3, Stress Free: n=4) commented that certain app components were mildly stressful (eg, the distraction games, background music). There were no reported adverse events experienced as a result of either intervention. Also, 7 participants commented that the Agoraphobia Free app was confusing to follow at certain points. Another participant explained that they could not use the treatment app as much as they wanted to because they had depression and did not feel motivated.

The results from this RCT showed that participants who received Agoraphobia Free did not improve more than those who received the Stress Free app. Both groups showed reductions in symptom severity over time that were statistically significant, but those reductions seemed to be equivalent across the 2 groups. Both treatment apps were safe to use and yielded similar completion rates. Moreover, completers of either intervention showed marked improvements in symptom severity, with a 10-point drop in the PAS at end point compared with baseline in the Agoraphobia Free group. However, findings from this analysis should be considered with caution, as we performed the analysis on a specific subgroup of participants, and there is the possibility that factors other than the intervention influenced the outcome (eg, participant expectations). In addition, the power of this analysis was very low (below 50%) due to the small sample size.

Throughout the trial, participant attrition was particularly high, as almost half of the participants recruited dropped out of the trial or stopped responding to emails. Dropout rates were similar to those reported in other Web-based trials of self-guided interventions [40,41]. Many participants dropped out after providing consent, which we had not expected. We had to exclude those participants from the analysis, as no baseline data were available. It is unlikely that this exclusion of participants could have led to bias, as participants at that point were unaware of the groups they were randomly allocated to, and thus attrition was random between the 2 groups. Similarly, participants who dropped out or stopped responding after receiving the intervention did not seem to differ from those who engaged in the trial, and attrition was equivalent across the 2 groups. The analytic strategy we chose is robust and uses all available data from each participant, producing less-biased results than other methods of analysis and data imputation [38]. Overall, the fact that participant attrition appeared random, in addition to the fact that we used data from all participants in the analysis, offer support and confidence in the validity of the study findings.

The trial had a few limitations. There was no follow-up after the completion of the intervention. Therefore, it was not possible to investigate whether the improvements in symptom severity were maintained over time or whether there were any differences between the 2 groups after a few months of using the apps. The information collected regarding the characteristics of the sample was also limited, as there were no data on comorbid disorders, other psychological or physiological treatments, or other demographic characteristics such as ethnicity and computer literacy. This information might have provided better insights into the sample and could have been related to intervention efficacy. There was also limited information on app completion, as the information available relied on participant report and we were not permitted to extract app use data for participants individually. Moreover, participants could not be blinded, which is a common limitation in eHealth trials.

Another limitation of the study was the absence of a waitlist control group. The comparison of 2 active groups that shared very similar features tested the effects of agoraphobia-specific therapeutic elements over and above those of a generic anxiety-related treatment. There was no control, however, for the effect of time, and it is therefore not possible to conclude whether the improvements observed were not because of natural recovery processes or factors other than the intervention. Although a waitlist control group would be a necessary addition in a future trial in order to clearly establish treatment efficacy, the primary focus of this study was to demonstrate whether a disorder-specific mobile-based intervention is warranted in the treatment of agoraphobic symptoms, compared with a more generic approach addressing anxiety.

Despite its limitations, the study also presented certain strengths. First, it is, to our knowledge, the first RCT to directly compare an agoraphobia-specific app with a generic anxiety-related app. Second, the sample recruited online was characterized by severe clinical symptoms and especially high levels of agoraphobia, which shows that recruitment based on self-identification is feasible and reliable. Such a sample may have also been hard to reach in a traditional multicenter trial with assessors conducting screening interviews, which would have been time consuming and would require a lot of resources. The trial was minimally intrusive for the participants, as it requested a limited amount of information, and the tools used were short and easy to complete. This helped achieve fair recruitment and response rates without any explicit individual guidance. Third, an important strength of the study was its external validity. The apps were offered as they would be offered in real-world settings, without requiring any screening of users or close supervision. There were very few restrictions on who could participate, therefore making our findings more easily generalizable to an Internet population who would simply download the app if they thought it would be relevant to them. Fourth, the results of the study provide an insight into what we would realistically expect from an unguided mobile intervention, such as a substantial dropout rate, small symptom change, and highest efficacy for those who consistently engage with and complete the intervention.

The findings of this study relate to previous studies on the treatment of anxiety disorders. The findings add to the evidence base of computerized cognitive behavioral interventions examined in anxiety disorders [21-23]. Our findings are also consistent with previous studies showing that completers of Web-based interventions for panic (with or without agoraphobia) benefit the most [40,42]. Importantly, the lack of a significant difference between the 2 groups indicates that a generic anxiety-related app and a diagnosis-specific app are equally effective in treating agoraphobic symptoms. This finding is in line with evidence supporting the efficacy of transdiagnostic (ie, unified, nonspecific) cognitive behavioral treatments in anxiety disorders. Transdiagnostic treatments are based on the premise that “commonalities across disorders outweigh the differences” [43]. A recent systematic review and meta-analysis showed that transdiagnostic treatments can be as efficacious as diagnosis-specific treatments in reducing anxiety symptoms and more effective than waitlist or attention control conditions [44]. Transdiagnostic CBT programs tested in different anxiety disorders (including panic disorder) can also be successfully administered via the Internet [45], producing effects equivalent to those of disorder-specific Internet-based interventions [46,47]. Moreover, transdiagnostic CBT does not differ from diagnosis-specific CBT in terms of treatment credibility [48]. Our study adds to this body of evidence suggesting that, for people with agoraphobia, an agoraphobia-specific app does not produce any additional benefits in relation to a transdiagnostic anxiety-targeting app.

The findings of this study have implications for clinical practice. This trial shows that mobile apps can be successfully administered to a particular population that is hard to reach otherwise, without requiring guidance by a clinician. This could potentially save time for clinicians, while it could also be more convenient for patients and help them overcome many barriers to treatment, such as traveling to sites or long waiting lists. Furthermore, a self-guided app could be easily introduced into a stepped care model as a minimal intervention, as it is less intense than a clinician-guided intervention or individual therapy [49]. Since there was no evidence of superiority of one type of intervention over the other in this study, a choice between a generic and a targeted approach could be ultimately based on clinical judgment or patient preference. However, as suggested by Norton and Barrera [48], a generic anxiety-targeting app might be more resource efficient and easier to implement and disseminate than a diagnosis-specific app.

Overall, this RCT suggests that transdiagnostic anxiety-targeting mobile apps can be as effective as disorder-specific apps for people with agoraphobic symptoms. Results show that individuals who identify as having agoraphobia do not benefit more from an agoraphobia-specific than from a transdiagnostic app. Future research conducted by independent research teams should replicate the results of this study, further investigating the possibility that mobile-based transdiagnostic interventions for anxiety can be as effective as current gold standard disorder-specific interventions. A trial with a larger sample size and a waitlist control group is warranted to establish intervention efficacy and cost effectiveness in this population. Additionally, the collection of more-extensive demographic and clinical information can help examine under which circumstances a diagnosis-specific or a generic approach is more appropriate. For example, future research could investigate whether patients with more severe symptoms or with comorbid disorders (eg, depression) benefit more from one type of intervention than the other. While there is still much to learn about treatment approaches in anxiety disorders, many studies, including this one, show that e-mental health interventions can overcome barriers and be effective in reducing clinical symptoms.