A full trial protocol is available elsewhere [38]. This trial was approved by the Medical Ethics Committee of the VU Medical Centre, Amsterdam (registration number 2010/77) and registered with the Dutch Trial Registry (NTR2233). A total of 481 participants who recently applied for psychological treatment at an outpatient clinic consented to be contacted by our research group and were referred to the researchers from August 2010 to December 2013. After briefing the participants about the aims of the study, screening, and obtaining informed consent in writing, eligible participants (n=212) were administered a telephone baseline questionnaire and participants were randomized to either the intervention group (n=105) or treatment as usual (n=107). Patients who were ineligible (n=111), declined participation, or could not be contacted (n=153) remained on the waiting list for face-to-face treatment. The research did not interfere with the outpatient clinics’ wait-list duration or start of treatment and participants could start face-to-face psychotherapy after the intervention or control group period.

All outcome measures were administered by phone at baseline (T0) and as self-assessment on the Internet at posttest (5 weeks from randomization, T1), using Web-based questionnaire software visibly associated to VU University Amsterdam. This relatively short period was selected to minimize the posttest assessments taking placing during face-to-face psychotherapy if face-to-face psychotherapy should incidentally take place earlier than 6 weeks after inclusion into the trial. To reduce study dropout, intensive reminder emails and telephone reminder calls were used for T1 assessments. Despite several email and telephone reminders, there was considerable variability in follow-up time (mean 50, SD 15.3 days), yet there was no significant difference in follow-up time between the intervention and wait-list control groups. All trial data were stored on a secured network complying with Dutch safety and privacy standards at the time of inclusion and accessible only to research staff. Data were anonymized as soon as possible.

Outcome measures are described in more detail elsewhere [38].

To obtain 90% statistical power with a 2-sided alpha equal to .05 and assuming a mean standardized effect size (Cohen’s d) of 0.7 in the intervention group and 0.2 in the control group, we calculated that 170 participants were needed to establish a clinical effect of the Internet intervention compared to wait-list controls. Assuming a dropout rate of 30% at 1-year follow-up, 244 participants should be included.

A computer-generated randomization table was prepared by a researcher not involved in the data collection (AvS). Randomization was stratified at clinic level and performed at a 1:1 ratio. To ensure approximately equal randomization ratios per clinic, blocks of 8 were used. An external researcher not involved in the project supervised a list of sequentially numbered allocations and assigned participants to the conditions. All project members involved in data collection were unaware of allocation status until randomization was definitive. Participants were enrolled by a master’s level research assistant.

Due to the nature of this trial, neither participants nor researchers could be blinded to treatment allocation. All outcome measures are self-report questionnaires, which makes blinding unnecessary.

Data were analyzed with SPSS for Windows, version 20 (IBM Corp, Armonk, NY, USA) according to the intention-to-treat (ITT) principle. Using the multiple imputation function implemented in SPSS 20, we imputed missing data at posttest yielding 50 imputed datasets with 50 iterations each using the multiple imputation option with predictive mean matching. Predictors for the imputing procedure were pretest and (nonmissing) posttest scores, as well as age, clinic, education level, gender, randomization status, and quality of life at pretest. Because SPSS does not automatically calculate pooled statistics for imputed datasets when using ANCOVA, we calculated these statistics by pooling the saved residuals from each imputed dataset and reported mean values and 95% confidence intervals. Between-group effects on the primary outcome measure (FQ) at posttest were calculated with an ANCOVA, with baseline scores of the FQ, BAI, and CES-D entered as a covariate. Within- and between-group effect sizes were reported as Cohen’s d. Effect sizes of d=0.2 are interpreted as small, effect sizes between 0.2 and 0.5 are interpreted as moderate, and effect sizes of 0.8 and upwards are interpreted as large. Due to the large amount of missing data at posttest, data are also presented separately for participants with full follow-up information. No interim analyses were performed. No stopping guidelines were postulated.

There were no changes from the published study protocol [38].

See Figure 3 for a flowchart and overview of participants in this trial. Of 481 participants assessed for eligibility, 212 were randomized to either intervention (n=105) or control (n=107). Baseline data are presented in Table 1. Apart from psychotropic medication use, there were no significant differences between intervention and control groups at baseline. Participants were mainly Dutch, female, and highly educated.

Posttreatment assessments were completed by 126 of 212 (59.2%) participants. Due to database corruption, 2 pretreatment assessments were unavailable. We tested whether there were significant differences in all baseline characteristics described in Table 1 between those who completed the follow-up assessments and those who did not, with dropouts scoring significantly higher on the BAI (t ₂₁₀=2.275, P=.02) and the CES-D (t ₂₁₀=2.489, P=.01), and dropouts being younger in age (t ₂₁₀=2.022, P=.04), less often highly educated (χ² ₂₁₀=8.1, P=.004), and taking psychotropic medication less often at the time of assessment (χ² ₂₁₀=35.4, P<.001). Intervention group participants were also more likely to be nonresponders (67%, 84/126 of completers were control group participants; χ² ₂₁₀=4.3, P=.04). Table 2 shows the statistically significant differences between nonresponders and study completers. The differences of other characteristics described in Table 1 were not statistically significant.

Of the 105 participants, 78 (76.4%) started using the intervention, 14 (13.3%) finished week 5, and 9 (8.8%) met the intended usage of 100% (all 8 exercises) in 5 weeks. Average adherence, as expressed as percentage of intended usage, was 37.5% (SD 30.7%); median number of exercises completed (out of a possible 8) was 3 (IQR 4.0). As found in a previous meta-analysis [52], higher education in this sample was associated with higher adherence (F _2,103=8.132, P=.005). Intervention participants were moderately positive about their coach (average grade 6.8 of 10, SD 1.06) and indicated that the quality of the feedback messages was satisfactory (10/43, 23%), good (18/43, 42%), or very good to excellent (13/43, 30%). The number of messages received was also evaluated as being balanced (not too many, not too few) by most participants (32/43, 74%). Mean scores for all 8 CSQ-8 items were acceptable (mean 2.78, SD 0.58-0.81; possible item range 1-4).

After imputing missing values at posttest and correcting for baseline scores of the FQ, BAI, and CES-D, ANCOVA showed a significant difference in FQ scores between intervention and control groups at posttest (F _2,208=6.327, 95% CI 5.977-6.686; P=.02, 95% CI .01-.02; partial η²=.030, 95% CI .03-.03). ANCOVAs also showed a significant difference in CES-D scores between intervention and control groups at posttest (F _2,208=6.121, 95% CI 5.550-6.669; P=.03, 95% CI .02-.03; partial η²=.029, 95% CI .03-.03), but no significant difference in BAI scores between intervention and control groups at posttest (F _2,208=4.097, 95% CI 3.818-4.376; P=.05, 95% CI .04-.06; partial η²=.020, 95% CI .02-.02). Changes in scores are presented in Table 3 and presented graphically in Figure 4.

After correcting for baseline scores and age (full follow-ups only), we found no significant differences in posttest scores between intervention and control groups for FQ (F _2,208=.137, P=.71, partial η²=.001), CES-D (F _2,208=2.086, P=.15, partial η²=0.017), or BAI (F _2,208=0.333, P=.57, partial η²=0.003).

No adverse events were reported or observed during the trial. In a recent trial, a few participants reported temporary adverse effects in an Internet-based intervention for social anxiety disorder [53]; in particular, exacerbation of anxiety symptoms and negative well-being. One control group participant (1/107, 0.5%) reported a worsening of complaints after having started the intervention. However, it is unknown whether this was due to the intervention or to other circumstances not related to this study.

Results show that an Internet-based guided intervention for phobic outpatients can be effective, with modest but significant effect sizes (d=0.28-0.35). The between-group effect sizes were small but significant in the imputed sample. Interestingly, effect sizes were similar for the FQ and CES-D. Because spontaneous recovery is common in depression [54] but not in phobia [5,6], it is unlikely that the decrease in phobic complaints can be attributed to spontaneous recovery rather than to the intervention. The intervention was not targeted specifically at depression; therefore, the significant decrease in depressive symptoms as measured by the CES-D might be attributable to spontaneous recovery rather than the intervention, although due to randomization this spontaneous recovery should probably have occurred in the control condition as well. Another possible explanation is the decrease in phobic complaints in the intervention group led to a commensurate decrease in depressive symptoms. Because the control group participants received a self-help book as compensation for their time invested in completing the baseline and follow-up assessments, there is a possibility that the between-group effect sizes are a more conservative estimate of the real effect size. Additionally, effect sizes for the BAI were lowest of all 3 primary outcome measures. This may be a result of the BAI being concerned mostly with those physical sensations often associated with panic attacks rather than phobias.

Compared with earlier research, recruitment of patients in the outpatient clinics seemed to be reasonably successful. Of the 481 patients assessed for eligibility, of 370 eligible patients we randomized 212 of the 244 participants intended. In all, 153 patients (31.8%) declined to participate in the study (before being screened) and 111 (23.1%) did not meet inclusion criteria. In total, we were able to include 44.1% of all referrals with approximately 2 patients needed to be screened to include a single participant and approximately 3 hours required for each included participant. This does, however, not include the amount of time needed at the outpatient clinic for administrative tasks. In contrast, a recent study of computerized CBT for anxiety in secondary care [55] managed to randomize only 8% (88 of 1141) of referrals. The recruitment percentage also compares rather favorably to trial recruitment in primary care, which can be problematic in terms of duration and numbers of participants recruited as compared to the planned recruitment time and numbers of participants [56]. For example, 1 study focusing on Internet-delivered CBT for depression managed to recruit only 7 patients from 11 general practices in 8 months [57], a marked contrast to our trial. This may be a result of keeping close contact with outpatient staff, although it is difficult to draw comparisons because similar studies with tight integration in routine outpatient clinics are scarce and health care systems differ on national levels. When comparing effect sizes of the current trial with earlier trials of Internet-based anxiety and phobia interventions, we found that the effect sizes of the current trial are low overall [19,58]. In face-to-face psychotherapy, effect sizes of psychological treatment for participants meeting diagnostic criteria were found to be lower than those not meeting the criteria (eg, [13]), but another study found large and sustained effects of Internet-based CBT for panic disorder with and without agoraphobia in routine psychiatric care [33]. Although these previous trials used CBT compared to exposure therapy, it should be noted that the intervention of the current trial included a number of CBT components and that there is no unambiguous evidence that CBT should outperform exposure therapy for phobias per se, with studies finding either no difference or a negligible advantage for either CBT of exposure therapy in phobias (eg, [14,19,59,60]). There is an exception for social anxiety disorder, where cognitive therapies appear to outperform exposure therapy [13,61,62]. It should be noted that just over half of the participants in the current trial were diagnosed with social anxiety disorder, indicating that perhaps greater attention should have been paid to the cognitive therapy elements in the intervention. However, comorbidity of different phobias was large in the current trial, and participants suffering from multiple phobias were encouraged to focus on a single, well-circumscribed area of phobic avoidance at the start of the intervention. Many participants explicitly chose to work on specific phobias or agoraphobia rather than social anxiety, indicating that exposure therapy was indeed the right choice for these participants.

Overall, few (n=9) participants completed all 8 exercises, and 14 completed all 5 weeks, indicating that some exercises were skipped by the participants. Ending a treatment early may not necessarily be a negative finding [63]. Although not many participants stated a reason for not finishing the intervention, the primary reasons given were a lack of time (8 of 27) and the intervention not being suited to the needs of the participant (5 of 27), both commonly cited barriers to the uptake of online interventions [64]. Despite the high acceptability, as expressed by the low percentage of participants refusing to enter the trial outright, low adherence to this intervention remains a cause for concern because it seems to be lower than generally found in other Internet interventions for anxiety disorders [64]. Possible causes for the lower adherence in this intervention may be that the intervention was too broad, targeting all types of phobias within 1 intervention. Additionally, participants were aware that they would receive face-to-face psychotherapy regardless of whether they finished the online intervention, which may have lowered their motivation to persist. Furthermore, the intervention itself may not have been persuasive enough to encourage repeated use [49]. Earlier efforts to identify predictors of treatment dropout in social anxiety disorder, for example, have not yielded consistent results [65], and future research into the causes of premature treatment termination is needed.

A particular strength of this study is the high acceptance rate among outpatients as compared with other studies in outpatients and specialized health care centers, which indicates that this sample is clinically relevant and that the results may generalize well across other outpatient samples using similar recruitment strategies. Implementation in routine practice would perhaps facilitate better uptake due to dropping the constraints surrounding research-oriented RCT setting (eg, randomization, filling in extra questionnaires).

A number of limitations should be taken into account when interpreting the results from this trial. Firstly, the number of participants fell slightly short from the target number of participants (212 randomized versus 244 targeted). Far-reaching cutbacks in Dutch mental health care during the recruitment into this trial resulted in a dwindling number of patients seeking help with outpatient clinics, effectively shrinking the overall participant pool. Secondly, recruitment through outpatient clinics depends on outpatient clinic staff and may be liable to selection bias. Although the included sample of participants seems relatively representative of a clinical sample, selection bias may have occurred during the outpatient clinics’ own selection procedures over which we had no influence. Thirdly, although we corrected for missing values at follow-up by using multiple imputation, the results should be interpreted with caution due to the large amount of missing data. Imputation is an approximation based on a combination of chosen predictors, techniques, and imputing algorithms, which may yield varying results in different datasets [66], making extensive sensitivity analyses time consuming and inconclusive at best. Although some argue that using covariates yields similar results to imputing [67], this may depend heavily on the dataset, and multiple imputation remains the solution of choice for missing data [68]. Regardless of the method for accounting for missing data, the large amount of missing data in this trial is a limitation and means that results should be interpreted with caution. Thirdly, there was considerable variability in the time between baseline assessment and follow-up (median 48 days, range 29-138 days). However, there were no significant correlations between baseline and posttest scores and the time in days between baseline assessment and posttest assessment, which indicates that the posttest scores as used are a representative assessment. Finally, offering the control group participants a self-help book may have influenced the between-group results because wait-list participants using the self-help book may have improved. However, because an improvement in control group participants would mean a smaller contrast between groups, this would lead to a more conservative estimate of treatment effects. Furthermore, it has been proposed that the use of a wait-list group design may not be a representative control group in that it is functionally different from a no-treatment group. It has also been put forward as actually being a “nocebo” group in that waitlisted participants actually do worse than no-treatment participants [69], which would theoretically lead to an inflation of between-group effect sizes. In the light of the current trial, however, both intervention and control groups were scheduled to receive face-to-face psychotherapy; as such, the arguments pertaining to possibly higher (or lower) effect sizes when using a wait-list control group do not necessarily apply to the current trial.

In summary, adding an online guided intervention to routine face-to-face treatment may prove beneficial for outpatients, regardless of type of phobia diagnosis. However, effect sizes were markedly lower (d=0.28-0.35) than those found in research on psychological treatment for phobias in the general population (d=0.70-1.84) [12-14] and for anxiety in primary care (d=0.57) [70], and were only found for the imputed ITT sample. Because there was systematic attrition in this trial, the significant differences between completers and dropouts may provide valuable information to identify focal points for targeted attrition reduction, although previous efforts to identify nontechnological factors influencing attrition in online interventions have yielded inconclusive results [71-75]. Given the combination of high acceptability and low adherence, future research should focus on optimizing the usability and persuasive design of this intervention to improve retention and adherence [49] to maximize potential benefits of an intervention that efficiently uses the time spent waiting for face-to-face psychotherapy. Independent replication of the current results in different outpatient settings and countries is needed to verify the findings before robust inferences can be made, but the current results are promising.