Published on 10.01.20 in Vol 22, No 1 (2020): January
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/16804, first published Oct 26, 2019.
Effectiveness and Cost-Effectiveness of a Self-Guided Internet Intervention for Social Anxiety Symptoms in a General Population Sample: Randomized Controlled Trial
Background: Many people are accessing digital self-help for mental health problems, often with little evidence of effectiveness. Social anxiety is one of the most common sources of mental distress in the population, and many people with symptoms do not seek help for what represents a significant public health problem.
Objective: This study aimed to evaluate the effectiveness of a self-guided cognitive behavioral internet intervention for people with social anxiety symptoms in the general population.
Methods: We conducted a two-group randomized controlled trial in England between May 11, 2016, and June 27, 2018. Adults with social anxiety symptoms who were not receiving treatment for social anxiety were recruited using online advertisements. All participants had unrestricted access to usual care and were randomized in a 1:1 ratio to either a Web-based unguided self-help intervention based on cognitive behavioral principles or a waiting list control group. All outcomes were collected through self-report online questionnaires. The primary outcome was the change in 17-item Social Phobia Inventory (SPIN-17) score from baseline to 6 weeks using a linear mixed-effect model that used data from all time points (6 weeks, 3 months, 6 months, and 12 months).
Results: A total of 2122 participants were randomized, and 6 were excluded from analyses because they were ineligible. Of the 2116 eligible randomized participants (mean age 37 years; 80.24%, 1698/2116 women), 70.13% (1484/2116) had follow-up data available for analysis, and 56.95% (1205/2116) had data on the primary outcome, although attrition was higher in the intervention arm. At 6 weeks, the mean (95% CI) adjusted difference in change in SPIN-17 score in the intervention group compared with control was −1.94 (−3.13 to −0.75; P=.001), a standardized mean difference effect size of 0.2. The improvement was maintained at 12 months. Given the high dropout rate, sensitivity analyses explored missing data assumptions, with results that were consistent with those of the primary analysis. The economic evaluation demonstrated cost-effectiveness with a small health status benefit and a reduction in health service utilization.
Conclusions: For people with social anxiety symptoms who are not receiving other forms of help, this study suggests that the use of an online self-help tool based on cognitive behavioral principles can provide a small improvement in social anxiety symptoms compared with no intervention, although dropout rates were high.
Trial Registration: ClinicalTrials.gov NCT02451878; https://clinicaltrials.gov/ct2/show/NCT02451878
J Med Internet Res 2020;22(1):e16804
Many people are accessing digital tools for self-help for a range of mental health problems . Social anxiety is one of the most common sources of mental distress in the population and represents a significant public health problem [ ]. It is characterized by a cluster of cognitive, behavioral, and physiological symptoms including an intense and persistent fear of being negatively evaluated in social or performance situations, along with avoidance of such situations. The individual fears that they may act in such a way, or show anxiety symptoms, which would lead to embarrassment or humiliation. A diagnosis of social anxiety disorder may be made when symptoms are persistent and lead to disruption of daily routine, and work or social life, or if the symptoms themselves cause marked distress. There is a spectrum of symptomatology in the general population, and even subclinical symptoms that do not reach a clinical diagnostic threshold can cause substantial impairment [ , ].
Effective psychological and pharmacological treatments exist for social anxiety symptoms, but many people with symptoms do not seek or receive these treatments [- ]. Self-guided digital tools have received much attention owing to their potential for high scalability and low marginal cost, in addition to the benefits of convenient access and anonymity they offer to people with social anxiety symptoms who may not seek help through more traditional routes because of embarrassment or fear of scrutiny [ ]. A 2014 meta-analysis of randomized controlled trials of unguided internet-based self-help for social anxiety disorder identified 5 studies (270 participants in total) showing evidence for effectiveness for these interventions with a pooled standardized mean difference of 0.66 (95% CI 0.39-0.94) [ ]. Subsequent trials using self-help interventions that use cognitive behavioral approaches have found similar effect sizes (between-group effect sizes ranging from 0.47 to 0.76) [ - ]. Previous studies were conducted on a relatively small scale (the largest number of participants in the intervention group in any previous individual study was 100 [ ]) and have generally been confined to cases of social anxiety of clinical severity, usually based on a structured interview assessment. Very little work has attempted to examine the value of unguided self-help in a real-world context, where individuals self-select as requiring help with symptoms that may not reach a clinical threshold but may be causing them some level of distress and choose to access digital tools themselves, with no clinician contact at all. In this study, we examine the effectiveness and cost-effectiveness of the self-help E-couch social anxiety tool (described in detail below). This was chosen as it is a self-directed online intervention based on cognitive behavioral therapy principles including components of known effectiveness in face-to-face therapy. A previous laboratory-based comparative study of the E-couch social anxiety tool with 21 participants (mainly university students) in the E-couch arm showed pretest to posttest improvement in social anxiety measures [ ].
We, therefore, undertook the first large-scale pragmatic randomized trial of an online self-guided cognitive behavioral intervention for people with self-reported social anxiety symptoms in the general population. Our experimental hypothesis was that participants who received the intervention would have a greater improvement in symptoms of social anxiety compared with participants who did not.
Trial Design and Participants
A two-arm, parallel-group randomized controlled trial was conducted to compare the effectiveness and cost-effectiveness of a Web-based and mobile-optimized self-guided intervention with a waiting list control condition for treating social anxiety symptoms. The study received ethics approval from the University of Oxford Medical Sciences Inter-Divisional Research Ethics Committee (MS-IDREC-C1-2015-167) and the Australian National University Human Research Ethics Committee (Protocol 2015/229) and is registered on ClinicalTrials.gov (NCT02451878). All participants provided informed consent to take part in the study using a self-completion online form. Outcomes were assessed at baseline, 6 weeks, 3 months, 6 months, and 12 months. The main follow-up time points were chosen to measure immediate effect (6 weeks as the intervention was designed to be used over a 6-week period) and long-term outcomes (12 months), along with interim time points (3 months and 6 months) to strengthen our repeated measures analysis and to support participant engagement. All study administration was conducted using automated online systems. The trial protocol is in.
Participants were recruited primarily through an online advertisement placed on the UK National Health Service (NHS) website. In addition, study advertisements seeking individuals with social anxiety symptoms were placed on university and charity websites and disseminated via email and social media. We aimed to capture people with a broad range of social anxiety symptoms in the general population, who were likely to be typical of those seeking help from self-directed digital tools. Interested potential participants completed an online screening questionnaire to assess eligibility. We excluded anyone currently receiving therapist-guided treatment for social anxiety disorder or who self-reported a diagnosis of schizophrenia or bipolar affective disorder. Initial inclusion criteria were having access to the internet-based intervention, aged 18 years or older, resident in England, having an email address and mobile telephone number (to receive study emails and text alerts), and an initial criterion of scoring in a subclinical range of 13 to 19 on the 17-item Social Phobia Inventory (SPIN-17). We had initially chosen the 13 to 19 range with expert advice as this would, in theory, capture those scoring above the population mean score (11-12) while excluding those scoring above the commonly used threshold of 19, which indicates further assessment may be warranted (although this threshold does not represent a diagnosis). However, early in recruitment, it became apparent that most people in the general population volunteering for this study scored much higher than this, and the distribution of SPIN-17 scores meant that very few scored in the low range. There was clear evidence of a high level of unmet need among individuals living with social anxiety symptoms in the community and not seeking help elsewhere. With advice from our independent Trial Steering Committee, we, therefore, revised and reregistered the protocol (in line with good practice in clinical trials) to modify the inclusion criteria to include all individuals scoring 13 or more on SPIN-17, therefore capturing those in our hypothesized subclinical range of 13 to 19, as well as those with a higher score. We continued to exclude anyone receiving professional help, and therefore, the final sample represented adults in the general population who self-reported some level of social anxiety symptoms but who were not receiving treatment for social anxiety. Potentially eligible participants completed consent, and 24 hours later, they were sent an email link to self-complete their baseline measures using online questionnaires.
Randomization and Masking
Once baseline measures had been completed, participants were randomized (1:1 ratio) to either the intervention group (E-couch) or the waiting list control group using a computer-generated random number sequence run through an automatic online program using a block size of 2 without stratification. Due to the nature of the intervention, participants were not blind to allocation.
Given that this was a general population sample, all participants continued to receive usual care. Participants in the intervention arm were given access to a password-protected website that contained the E-couch social anxiety module. The website was mobile-optimized and could, therefore, be used on a smartphone with the look and feel of a dedicated app, or on a computer browser.
The E-couch social anxiety module is a self-directed interactive program based on cognitive behavioral therapy principles. The program is divided into 6 modules: a literacy section, which provides information about the symptoms of social anxiety, types of available help, and effective treatments, and 5 toolkits comprising exposure practice, cognitive restructuring (modifying your thinking), attention practice, social skills training, and relaxation. The content of the toolkits consists of evidence-based information, interactive exercises, and workbooks based on cognitive behavioral principles; participants could complete the modules in any order. Participants were advised to access and use the intervention over the initial period of 6 weeks (although they could work through the intervention at their own pace and were able to access it for the full 12-month duration of the study). “Ideal” usage of the intervention would entail engagement with 1 new module each week and ongoing updates to diaries and workbooks based on the user’s real-life experiences. E-couch was developed by the ehub team at the Australian National University National Institute for Mental Health Research. The intervention was adapted for this study to create a “stand-alone” social anxiety intervention that was accessed via a password-protected portal and with the usual E-couch branding removed. The program was adapted for a UK audience by removing Australian-specific terminology and undertaking user testing on the new version. No changes were made to the intervention during the study period.
Waiting List Control
Participants in the control group were informed that they had been put on a waiting list to receive access to the intervention in 12 months. They were asked to complete baseline and follow-up measures at the same time as participants in the intervention group. They received no other intervention.
Automated text (SMS) message and email reminders were sent to participants in both groups to reduce attrition. Participants in the intervention condition received 1 text message within 24 hours of randomization to remind them to access the intervention and 3 email reminders during the 6-week intervention period to remind them to log in to access the program. In addition, all participants received email invitations to complete follow-up surveys at each outcome measure time point, with those who failed to complete receiving a reminder email followed by a reminder SMS text message.
The primary outcome was the change in SPIN-17 score from baseline to 6 weeks. The SPIN-17 is a 17-item self-rated scale covering the main social anxiety symptoms of fear, avoidance, and physiological disturbance. The responses to 17 statements (such as “I avoid talking to people I don’t know”) are rated on a 5-point scale from “not at all” (score=0) to “extremely” (score=4) to indicate the extent to which each statement reflects how the respondent was feeling in the past week, with higher scores reflecting greater social anxiety symptoms. The SPIN-17 has good test-retest reliability, internal consistency, and convergent and divergent validity . Secondary outcomes were all also self-report measures with good reliability and validity: the 8-item Brief Fear of Negative Evaluation (BFNE-S scale), which is very commonly used in studies of social anxiety and measures one of the key psychological constructs of social anxiety (example item: “I am frequently afraid of other people noticing my shortcomings”) [ ]; the 20-item Centre for Epidemiologic Studies Depression scale (CES-D), which has been widely used in online studies of anxiety and depression interventions to measure depressive symptoms [ ]; the 7-item Short Warwick-Edinburgh Mental Well-Being Scale (SWEMWBS), a measure of mental well-being requiring participants to provide the extent of their agreement with statements about thoughts and feelings over the previous 2 weeks, which has been shown to be responsive to change (example item: “I’ve been feeling useful”) [ , ]; and the widely used and validated 36-item Short Form Health Survey (version 1) to measure health status and quality of life expressed in mental and physical component scores [ ]. We also measured usage of the intervention in terms of number of E-couch modules completed, total time in minutes spent on modules, and total page views. Adverse events were not anticipated, but participants were asked to self-report any ill effects thought to be related to the intervention.
We aimed to recruit 2104 participants (ie, 1052 per group) to this trial, to detect a small between-group standardized mean difference of 0.2 at 5% two-sided significance level and 90% power, assuming a high level of potential attrition of up to 50% given the fully self-guided nature of the intervention and automated nature of the trial (all trial procedures were conducted online). Although previous studies have suggested a larger treatment effect for internet-delivered interventions, we believed this treatment effect is too optimistic for a pragmatic trial of a self-guided treatment in a general population sample. The target effect size, although small at an individual level, can potentially translate into an important population-level change .
The statistical analysis was finalized before unblinding of the data. Primary analysis was modified intention to treat according to allocated group irrespective of adherence and with at least one outcome questionnaire completed post randomization. A linear mixed-effect model was fitted to the primary outcome data, using data collected at 6 weeks, 3 months, 6 months, and 12 months. Participant was included as a random intercept. Randomized group, baseline SPIN-17 score, time, and time by randomized group interaction term were fitted as fixed effects. An unstructured variance covariance matrix was specified between repeated measures on the same individual. Assumptions of normality and constant variance for linear mixed-effects models were assessed by residual plots and other diagnostics plots.
Given the high level of attrition expected in online trials of self-guided digital interventions, we also prespecified a Complier Average Causal Effect (CACE) analysis for the primary outcome and 2 other main outcomes, to include only participants who completed at least one module of the intervention and at least one outcome assessment to investigate the effect of the intervention in participants who adhered to the intervention. An instrumental variable approach was adopted to provide the CACE estimate at 6 weeks. This method involved a 2 least squares (using the “ivregress 2sls” command in STATA SE Version 15.1, StataCorp, Texas) estimation from fitting a linear regression model of the primary outcome, adjusting for baseline SPIN-17 and compliance instrumented on randomized group .
Similar approaches were undertaken for other outcomes. A CACE analysis at 6 weeks was conducted on fear of negative evaluation (BFNE-S) and mental well-being (SWEMWBS) measures, similarly adjusting for baseline BFNE-S and SWEMWBS in the linear regression models. Safety analyses were not conducted as there were no adverse events reported during the study period.
In anticipation of high levels of dropout, we prespecified various sensitivity analyses to explore the impact of assumptions regarding missing data in the primary outcome analysis. These included analyses (1) of participants with complete data at all time periods, (2) adjusting for factors found to be predictive of missingness, (3) fitting a pattern mixture model to assess different degrees of missing not at random, as well as (4) an assessment of missing not at random assumption for the primary outcome by assuming plausible arm-specific differences of missing SPIN-17 score between responders (with SPIN-17 score at 6 weeks) and nonresponders (missing SPIN-17 score at 6 weeks) [, ].
Predefined subgroup analyses were conducted on change at 6 weeks for SPIN-17, BFNE-S, CES-D, and SWEMWBS for baseline SPIN-17 (<19, ≥19) to ascertain if the benefit differed between groups scoring above or below the screening threshold and for certain demographic characteristics to determine if the effectiveness of the intervention differed by the individual characteristics we had measured, that is, age (≤35, >35 years), gender (male, female), educational level (degree, no degree), and ethnicity (any white, nonwhite). Subgroup analyses were conducted by inclusion of an interaction term of baseline subgroup by randomized group by time in the linear mixed model. Descriptive statistics were used to describe usage data, adherence, and self-reports of other help received during the study period. All statistical analyses were performed using STATA SE version 15.1 .
A cost utility analysis from an NHS and social care perspective was conducted within this trial to assess the cost-effectiveness of the intervention. The total costs of developing, modifying, delivering, and maintaining the intervention were obtained, and the mean intervention cost was estimated for the participants recruited in the intervention group. Data on health care service utilization (for any reasons) were collected for all participants, including primary care consultations, hospital outpatient appointments, and hospital admissions. Unit costs for these health services were obtained from the Personal Social Services Research Unit (2016-2017) using national average costs . Maximum follow-up was 1 year; therefore, no discounting was applied. The total and mean costs for the intervention and the waiting list control group were calculated. Effectiveness was measured in quality-adjusted life years (QALYs) using the under-the-curve approach by combining the duration of follow-up with the health status utilities at the start and end points. Health status was measured using the self-reported SF-36 measure at baseline, 6 weeks, 3 months, 6 months, and 12 months. The analysis examined short-term (6 weeks) and long-term (12 months) impact. Health status utilities were converted from SF-36 to SF-6D indices using the established UK-based utility algorithm obtained through the University of Sheffield Licensing [ ]. The primary outcome was the incremental cost per QALY gained between the intervention group at 6 weeks and 12 months.
Participant Characteristics and Trial Flow
Recruitment took place between May 11, 2016, and May 9, 2017, when the target sample size was reached. Participants were followed up for 1 year. Final data were locked on June 27, 2018 (allowing time for delayed 12-month follow-up responses).shows the flow diagram of the participants throughout the study period. We screened 9447 participants of whom 5932 (62.79%) were ineligible, and a further 1393 (14.74%) did not complete the baseline measures. We randomized 1061 (1061/2122, 50.00%) participants to E-couch and 1061 (1061/2122, 50.00%) to the control group. A total of 6 participants who were randomized to the study were excluded from all analyses because their later responses indicated they did not meet the inclusion criteria in terms of age, leaving 2116 participants randomized and included in analyses. shows the baseline characteristics, which were similar across both groups. Owing to a software error, many participants were not sent the email requesting completion of their interim (3 months or 6 months) outcome measures. This error did not affect emails sent at the main follow-up time points of baseline, 6 weeks, and 12 months, and data from all time points were included in the analysis. Attrition rates differed significantly between groups with an overall loss to follow-up at the main follow-up time point (6 weeks) of 42.9%, with a loss of 60.8% in the intervention arm and 25.3% in the control arm (see and ). By 12 months, the primary outcome was available for 349 of 1061 (32.89%) participants in the intervention group and 710 of 1061 participants (66.92%) in the control group.
|Baseline characteristics||E-couch (n=1058)||Control (n=1058)||Total randomized (N=2116)|
|Age (years), mean (SD); range||37.4 (13.9); 18-84||36.9 (13.6); 18-82||37.17 (13.8); 18-84|
|Gender, n (%)|
|Female||859 (81.19)||839 (79.30)||1698 (80.25)|
|Male||187 (17.67)||213 (20.13)||400 (18.90)|
|Other||10 (0.95)||6 (0.57)||16 (0.76)|
|Missing||2 (0.19)||0 (0.00)||2 (0.09)|
|Marital status, n (%)|
|Married or in a civil partnership||327 (30.91)||311 (29.40)||638 (30.15)|
|Not married||727 (68.71)||745 (70.42)||1472 (69.57)|
|Missing||4 (0.38)||2 (0.19)||6 (0.28)|
|Education, n (%)|
|Degree||514 (48.58)||538 (50.85)||1052 (49.72)|
|No degree||544 (51.42)||520 (49.15)||1064 (50.28)|
|Employment status, n (%)|
|Employed||610 (57.66)||623 (58.88)||1233 (58.27)|
|Unemployed||440 (41.59)||426 (40.26)||866 (40.93)|
|Missing||8 (0.76)||9 (0.85)||17 (0.80)|
|Income (£), n (%)|
|≤25,000||736 (69.57)||739 (69.85)||1475 (69.71)|
|>25,000||322 (30.43)||319 (30.15)||641 (30.29)|
|Ethnicity, n (%)|
|White||917 (86.67)||918 (86.77)||1835 (86.72)|
|Nonwhite||141 (13.33)||140 (13.23)||281 (13.28)|
|Social Phobia Inventory-17, mean (SD); range||39.6 (13.1); 13-68||39.8 (13.4); 13-68||39.7 (13.3); 13-68|
|Mental well-being score (Short Warwick-Edinburgh Mental Well-Being Scale), mean (SD); range||17.7 (3.1); 7.0-30.7||17.7 (3.3); 7.0-35.0||17.7 (3.2); 7.0-35.0|
|Brief Fear of Negative Evaluation score, mean (SD); range||22.5 (7.1); 0-32||22.4 (7.3); 0-32||22.4 (7.2); 0-32|
|Centre for Epidemiologic Studies Depression scale, mean (SD); range||30.5 (12.3); 2-60||30.7 (12.2); 0-58||30.6 (12.2); 0-60|
|Short Form-36 (physical component summary), mean (SD); range||50.2 (10.3); 12.2-69.3||49.8 (10.6); 49.8 (10.6)||50.0 (10.4); 12.2-69.3|
|Short Form-36 (mental component summary), mean (SD); range||49.9 (9.5); 29.3-77.1||50.1 (9.4); 30.8-83.9||50.0 (9.5); 29.3-83.9|
Over the study period, there was a reduction of social anxiety symptoms in the E-couch group compared with that in the control group (see). At 6 weeks, the E-couch group had a mean (SD) reduction of SPIN-17 score of −6.2 (10.8) and the control group −3.99 (9.3). The adjusted mean difference (95% CI; P value) in change in SPIN-17 score in E-couch compared with control was −1.94 (−3.13 to −0.75; P=.001; ). This equates to a standardized mean difference effect size (between groups) of 0.2 (the pooled SD for SPIN-17 change was 9.81). At the 6-week follow-up, SPIN-17 outcome measures were available for 415 (415/1064, 39.00%) and 790 participants (790/1064, 74.25%) in the E-couch and control groups, respectively. In the CACE analysis, adjusted mean difference (95% CI; P value) in change in SPIN-17 score for intervention compared with control was −2.95 (−4.30 to −1.61; P<.001; ). The results from the sensitivity analyses undertaken to explore missing data assumptions were also consistent with the primary outcome 6-week findings. These included analyses that only considered data from completers (defined as participants who returned all their outcome measures at the main time points of baseline, 6 weeks, and 12 months; see ) and the findings of the pattern mixture model even when assuming different missing data patterns in the intervention or control group. Finally, findings were also similar to the primary outcome analysis even under the assumption of missing not at random.
|Outcomes||Mixed-effect model analysis||Complier average causal effect analysis|
|Adjusted difference in mean change (95% CI)||P value||Adjusted difference in mean change (95% CI)||P value|
|Social Phobia Inventory-17a|
|E-couch vs control (6 weeks)||−1.94 (−3.13 to −0.75)||.001||−2.95 (−4.30 to −1.61)||<.001|
|E-couch vs control (12 months)||−3.07 (−4.32 to −1.82)||<.001||N/Ab||N/A|
|Brief Fear of Negative Evaluation score|
|E-couch vs control (6 weeks)||−1.09 (−1.79 to −0.38)||.003||−1.60 (−2.38 to −0.82)||<.001|
|E-couch vs control (12 months)||−2.33 (−3.08 to −1.58)||<.001||N/A||N/A|
|Short Warwick-Edinburgh Mental Well-Being Scale|
|E-couch vs control (6 weeks)||0.38 (−0.02 to 0.77)||.06||0.59 (0.17 to 1.02)||.006|
|E-couch vs control (12 months)||0.82 (0.39 to 1.24)||.001||N/A||N/A|
|Centre for Epidemiologic Studies Depression scale|
|E-couch vs control (6 weeks)||−3.35 (−4.54 to −2.15)||<.001||N/A||N/A|
|E-couch vs control (12 months)||−1.79 (−3.06 to −0.52)||.006||N/A||N/A|
|Short Form-36(physical component summary)|
|E-couch vs control (6 weeks)||0.398 (−0.41 to 1.20)||.33||N/A||N/A|
|E-couch vs control (12 months)||0.003 (−0.85 to 0.85)||.99||N/A||N/A|
|Short Form-36(mental component summary)|
|E-couch vs control (6 weeks)||1.06 (0.12 to 1.98)||.03||N/A||N/A|
|E-couch vs control (12 months)||2.06 (1.07 to 3.06)||<.001||N/A||N/A|
bN/A: not applicable.
shows the results for the secondary outcomes. At the 12-month follow-up, participants randomized to the E-couch group continued to show a greater reduction in severity of social anxiety symptoms than the control participants, with a mean (95% CI; P value) adjusted difference in change in SPIN-17 score of −3.07 (−4.32 to −1.82; P<.001; ). As with the primary outcome, the results of the sensitivity analyses exploring missing data assumptions were consistent with the main analysis SPIN-17 findings at 12 months ( ). The findings for the other outcome measures of fear of negative evaluation (BFNE-S), mental well-being (SWEMWBS), depression (CES-D), and the mental component scale of the SF-36 all showed statistically significant small improvements favoring E-couch compared with control (see ). There was no evidence of difference between groups for the physical component scale of the SF-36. All distributions of residuals from the fitted models satisfied the normality assumption. No adverse events were reported during the study period.
Usage Data and Subgroup Analyses
At 6 weeks, the mean (SD) number of E-couch modules fully completed (out of 6) was 1.87 (1.43), total mean (SD) time in minutes spent on modules was 35.3 (48.1), and total mean (SD) page views was 37.6 (41.3). Greater adherence to the intervention was not associated with baseline SPIN-17 score, age, gender, or ethnicity (). At 6 weeks, higher total page views or longer duration spent on modules was associated with larger improvement in social anxiety symptoms ( ). At 6 weeks, E-couch had a significantly greater impact in improving social anxiety symptoms for participants with baseline SPIN-17 score greater than 19 (usually taken as cutoff to indicate clinical assessment warranted) compared with the few participants scoring in the lower range (SPIN-17 score 13-19; P=.01; ). In this subgroup analysis, the lower SPIN-17 scorers (the ones we had originally defined as a subclinical population) had no benefit from the intervention compared with the control group. The E-couch intervention also had a significantly greater beneficial impact on depressive symptoms at 6 weeks in participants with higher baseline SPIN-17 scores (P=.007), and again in this subgroup analysis, the few participants scoring in the lower SPIN-17 range had no benefit on depressive symptoms compared with the control group. There was no evidence of heterogeneity in the effects of intervention for the subgroup analyses involving BFNE-S and SWEMWBS.
At both 6-week and 12-month follow-ups, the waiting list control group, in general, used more health care services than the E-couch group (seeand ). This resulted in a mean health care cost saving of £26.48 at the 6-week follow-up and £65.04 at the 12-month follow-up. Adding the mean intervention cost of £48.40 to the intervention group, the E-couch group cost more than the control group at 6 weeks but is cost saving at 12 months. In the cost utility analysis, at both 6-week and 12-month follow-ups, there were very small improvements of general health status in both the E-couch group and the waiting list control group, with the E-couch group improvement slightly more than the control group: the SF-6D indices increased from 0.6 at baseline to 0.64 at 6 weeks and 0.66 at 12 months for the E-couch group and from 0.6 at baseline to 0.62 at 6 weeks and 0.64 at 12 months for the waiting list control group (see ). At the 6-week follow-up, mean QALYs were 0.072 for the E-couch group and 0.070 for the control group, giving very small QALYs gains of 0.002 for the intervention over the control group. At the 12-month follow-up, mean QALYs were 0.635 for the E-couch group and 0.621 for the control group, with QALYs gain of 0.024 for the intervention over the control group. The incremental cost per QALY gained at 6 weeks was £10,960, which is highly likely to be cost-effective using accepted thresholds. At the 12-month follow-up, the E-couch dominated the waiting list control with more QALYs gained and less costs. Taking into consideration societal costs because of sick leave from work, the E-couch intervention was cost saving at both 6-week and 12-month follow-ups and, therefore, dominated the waiting list control.
|Group||General practitioner attendance costs, mean (SD)||Outpatient attendance costs, mean (SD)||Inpatient costs, mean (SD)||Cost of work days lost to sick leave, mean (SD)||Mean health care cost (SD)||Mean societal cost (SD)|
|E-couch (n=383)||38 (89.52)||45.43 (123.99)||72.77 (461.03)||106.65 (476.99)||156.19 (527.42)||264.26 (764.82)|
|Waiting list (n=761)||38.55 (65.89)||42.49 (111.43)||101.64 (570.51)||123.78 (477.48)||182.67 (643.67)||308.38 (871.52)|
|Group||General practitioner attendance costs, mean (SD)||Outpatient attendance costs, mean (SD)||Inpatient costs, mean (SD)||Cost of work days lost to sick leave, mean (SD)||Mean health care cost (SD)||Mean societal cost (SD)|
|E-couch (n=324)||101.10 (152.66)||117.61 (319.44)||207.92 (864.18)||379.47 (1740.74)||425.30 (1077.82)||806.08 (2198.79)|
|Waiting list (n=680)||106.34 (198.02)||141.84 (416.68)||242.16 (1000.48)||375.18 (1248.74)||490.34 (1264.51)||869.43 (1823.07)|
|Group||Baseline||6 weeks||12 months|
|SF-6Da, mean (SD)||SF-6D, mean (SD)||QALYb, mean (SD)||SF-6D, mean (SD)||QALY, mean (SD)|
|E-couch||0.60 (0.10)c||0.64 (0.12)d||0.072 (0.012)d||0.66 (0.12)e||0.635 (0.10)e|
|Waiting list||0.60 (0.10)c||0.62 (0.11)f||0.070 (0.011)f||0.64 (0.12)g||0.621 (0.10)g|
aSF-6D: six-dimensional health state short form.
bQALY: quality-adjusted life year.
Our findings showed that this fully self-guided internet intervention gave a small reduction in social anxiety symptoms in participants recruited online from the general population, compared with a usual care waiting list control group, and this small but positive finding was robust to the sensitivity analyses, which explored our missing data assumptions. There was a similarly small but significant improvement in fear of negative evaluation. These improvements were also found in the CACE analyses and maintained at the 12-month follow-up. In the context of a very common mental health problem, this finding suggests that automated self-help delivered via the internet could reduce the overall level of social anxiety symptoms in the population, although at an individual level, the mean symptomatic benefit is small (d=0.2). The study findings provide no evidence as to whether this fully self-guided approach has a role in a clinical setting, where, to date, the evidence base suggests that although unguided self-help has effectiveness, therapist-guided and therapist-led approaches are likely to be superior. The cost-effectiveness analysis demonstrated that the intervention is likely to be cost-effective in both the short and long term, although the gain in general health status and QALY score was very small. The benefit seen in the condition-specific social anxiety outcome measures was greater than the general health status used in the cost utility analysis. Furthermore, the intervention cost could be substantially reduced if the E-couch is used by large numbers at a population level as a public health tool. Given that many people with social anxiety symptoms do not seek help, and that therapist-supported approaches are limited in supply, the findings suggest that unguided digital intervention for social anxiety can be beneficial for some people who do not access professional help and who are increasingly seeking support from apps and other digital tools. The self-help approach tested here might also complement face-to-face therapy, potentially reducing the amount of therapist contact time required and perhaps helping to maintain engagement, although these suggestions need to be empirically tested in future effectiveness and cost-effectiveness work.
Comparison With Prior Work
This study adds to the body of work showing small positive effects for unguided digital self-help for social anxiety [, - ] and a range of other mental health problems [ ]. Our effect size is smaller than that reported by others. Previous studies have had far fewer participants and usually required them to meet diagnostic criteria for social anxiety disorder. Our aim was to undertake a pragmatic trial addressing social anxiety symptoms (rather than disorder) among individuals in the general population. Our broad inclusion criteria, recruiting volunteers from the general population through internet adverts, including those with symptoms not reaching a diagnostic threshold, are likely to have contributed to the more modest benefit compared with previous work. We made the additional decision to conduct the trial in a fully automated and naturalistic way with no researcher contact to encourage intervention use. Our approach was intended to reflect the real-world situation of members of the public self-selecting digital tools and using them with no contact with health services.
Strengths and Limitations
This study exemplifies both the strengths and weaknesses of undertaking online trials for digital interventions. We were able to recruit large numbers of participants from the general population using digital advertising, and we were able to deliver all measures and the intervention remotely, using little resources and with no requirement for any “real-world” contact between participants and researchers. The flipside of this was that, in common with other fully automated trials of unguided online interventions, there were high levels of dropout from the intervention and attrition from the trial . This is commonly seen in internet research [ ], including the higher level of retention in the control arm [ ], which may be partly explained by these participants being on a waiting list and, therefore, having an incentive to keep returning to complete measures, and partly by participants in the intervention arm being required to “take action” (work through the intervention), whereas the control group could be more passive. Other possible reasons for dropout include some participants not liking the intervention, or feeling it was not working, or indeed dropping out because they felt they had improved and no longer needed it. The high loss to follow-up was compounded at the 3- and 6-month follow-up points by a software glitch, which reduced the number of emails sent to participants at this time. Fortunately, these were always intended as interim time points measured to contribute to the overall mixed linear model. We undertook sensitivity analyses and explored various approaches to adjusting for the missing data. All outcome measures were self-report with no observer-rated objective assessment. This was in line with our desire to deliver a fully automated trial, and the scales are well validated, but the subjective nature of these measures is a potential source of response bias. We did not employ a placebo but instead used a waiting list comparator whereby people received “usual care.” In other work, educational website placebos have often demonstrated an active effect [ ]. Our pragmatic choice of control group, given that participants were not blind to allocation, may have introduced bias and increased the likelihood of a beneficial effect. Finally, most participants in this study were women. Social anxiety symptoms are twice as common in women than men [ ], and women are more likely to seek health care generally [ ]. Further work on the predictors and mediators of both adherence and response would be valuable [ , ].
For people with social anxiety symptoms in the general population who are not receiving other forms of help, an online unguided tool based on cognitive behavioral principles accessed via a computer or mobile phone gave a small but significant improvement in social anxiety symptoms compared with no intervention. As with many online trials of digital interventions, we experienced a “methodological trade-off” between having a cheap, scalable model of intervention delivery versus the statistical challenge of a high degree of missing data. Our findings suggest this intervention could potentially offer the first self-help rung on the ladder of a stepped approach to social anxiety symptoms.
The authors would like to thank the independent members of our Trial Steering Committee: Dr Peter Davidson MBBS, MSc (Chair), formerly Director of the National Institute for Health Research Evaluation, Trials and Studies Coordinating Centre, Chilworth, Southampton, UK; Dr Ceire Costelloe, PhD, Director of Global Digital Health Unit, Imperial College London, UK; Dr Adam Geraghty, PhD, Faculty of Medicine, University of Southampton, Southampton, UK; and Mr Gareth Stephens, lay representative. They would also like to thank Mr Anthony Bennett, the Chief Technology Officer at ehub Health, part of the team who developed and implemented the trial software, for technical expertise and support. The authors would also like to thank the participants who took part in this study.
This work was funded by the MQ: Transforming Mental Health charity under its PsyIMPACT funding call (MQ14PE_25). JP is also funded by the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care Oxford at Oxford Health NHS Foundation Trust. YY is supported by the National Institute for Health Research Oxford Biomedical Research Centre. KG was supported by a National Health & Medical Research Council Senior Research Fellowship.
The funding bodies had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, and approval of the manuscript; and decision to submit the manuscript for publication. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.
JP had full access to all study data and takes responsibility for the integrity of the data and the accuracy of the data analysis. All authors were responsible for study concept; design, acquisition, analysis, or interpretation of data; critical revision of the manuscript for important intellectual content; and administrative, technical, or material support. JP was responsible for drafting of the manuscript. JM, MS, and LY were responsible for statistical analysis. YY was responsible for economic evaluation. JP and KG were responsible for obtaining funding. JP and KG were responsible for study supervision.
Conflicts of Interest
All authors have completed and submitted the International Committee of Medical Journal Editors Form for Disclosure of Potential Conflicts of Interest. ehub Health Pty Ltd, a spinout from the Australian National University (ANU) has been granted the license to the E-couch intervention with royalties returning to the ANU. KB is an owner and employee of ehub Health Pty Ltd. As a cocreator of E-couch, KG is entitled to a percentage of any royalties received by the ANU from ehub Health Pty Ltd. She has no other financial interest in ehub Health Pty Ltd but is an honorary scientific advisor to the company. JP is guarantor of this paper and has no interests to declare in the E-couch intervention and KB and KG were not involved in the statistical analysis. No other disclosures were reported.
Study protocol.DOCX File , 733 KB
Sensitivity, subgroup and additional analyses.DOCX File , 256 KB
Descriptive statistics of Social Phobia Inventory (SPIN-17) score at all time points by randomized group.DOCX File , 18 KB
CONSORT‐EHEALTH checklist (V 1.6.1).PDF File (Adobe PDF File), 394 KB
- Peiris D, Miranda JJ, Mohr DC. Going beyond killer apps: building a better mHealth evidence base. BMJ Glob Health 2018;3(1):e000676 [FREE Full text] [CrossRef] [Medline]
- Bandelow B, Michaelis S. Epidemiology of anxiety disorders in the 21st century. Dialogues Clin Neurosci 2015 Sep;17(3):327-335 [FREE Full text] [Medline]
- Crisan LG, Vulturar R, Miclea M, Miu AC. Reactivity to social stress in subclinical social anxiety: emotional experience, cognitive appraisals, behavior, and physiology. Front Psychiatry 2016;7:5 [FREE Full text] [CrossRef] [Medline]
- Fehm L, Beesdo K, Jacobi F, Fiedler A. Social anxiety disorder above and below the diagnostic threshold: prevalence, comorbidity and impairment in the general population. Soc Psychiatry Psychiatr Epidemiol 2008 Apr;43(4):257-265. [CrossRef] [Medline]
- Mayo-Wilson E, Dias S, Mavranezouli I, Kew K, Clark DM, Ades AE, et al. Psychological and pharmacological interventions for social anxiety disorder in adults: a systematic review and network meta-analysis. Lancet Psychiatry 2014;1(5):368-376. [CrossRef]
- Cuijpers P, van Straten A. Improving outcomes in social anxiety disorder. Lancet Psychiatry 2014 Oct;1(5):324-326 [FREE Full text] [CrossRef] [Medline]
- Wang PS, Berglund P, Olfson M, Pincus HA, Wells KB, Kessler RC. Failure and delay in initial treatment contact after first onset of mental disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2005 Jun;62(6):603-613. [CrossRef] [Medline]
- Boettcher J, Carlbring P, Renneberg B, Berger T. Internet-based interventions for social anxiety disorder - an overview. Verhaltenstherapie 2013;23(3):160-168. [CrossRef]
- Ivanova E, Lindner P, Ly KH, Dahlin M, Vernmark K, Andersson G, et al. Guided and unguided Acceptance and Commitment Therapy for social anxiety disorder and/or panic disorder provided via the internet and a smartphone application: a randomized controlled trial. J Anxiety Disord 2016 Dec;44:27-35. [CrossRef] [Medline]
- Dear BF, Staples LG, Terides MD, Fogliati VJ, Sheehan J, Johnston L, et al. Transdiagnostic versus disorder-specific and clinician-guided versus self-guided internet-delivered treatment for Social Anxiety Disorder and comorbid disorders: a randomized controlled trial. J Anxiety Disord 2016 Aug;42:30-44 [FREE Full text] [CrossRef] [Medline]
- Gershkovich M, Herbert JD, Forman EM, Schumacher LM, Fischer LE. Internet-delivered acceptance-based cognitive-behavioral intervention for social anxiety disorder with and without therapist support: a randomized trial. Behav Modif 2017 Sep;41(5):583-608. [CrossRef] [Medline]
- Berger T, Urech A, Krieger T, Stolz T, Schulz A, Vincent A, et al. Effects of a transdiagnostic unguided internet intervention ('velibra') for anxiety disorders in primary care: results of a randomized controlled trial. Psychol Med 2017 Jan;47(1):67-80. [CrossRef] [Medline]
- McCall HC, Richardson CG, Helgadottir FD, Chen FS. Evaluating a web-based social anxiety intervention among university students: randomized controlled trial. J Med Internet Res 2018 Mar 21;20(3):e91 [FREE Full text] [CrossRef] [Medline]
- Kählke F, Berger T, Schulz A, Baumeister H, Berking M, Auerbach RP, et al. Efficacy of an unguided internet-based self-help intervention for social anxiety disorder in university students: a randomized controlled trial. Int J Methods Psychiatr Res 2019 Jun;28(2):e1766. [CrossRef] [Medline]
- Bowler JO, Mackintosh B, Dunn BD, Mathews A, Dalgleish T, Hoppitt L. A comparison of cognitive bias modification for interpretation and computerized cognitive behavior therapy: effects on anxiety, depression, attentional control, and interpretive bias. J Consult Clin Psychol 2012 Dec;80(6):1021-1033 [FREE Full text] [CrossRef] [Medline]
- Connor KM, Davidson JRT, Churchill LE, Sherwood A, Foa E, Weisler RH. Psychometric properties of the Social Phobia Inventory (SPIN). New self-rating scale. Br J Psychiatry 2000 Apr;176:379-386. [CrossRef] [Medline]
- Carleton RN, Collimore KC, McCabe RE, Antony MM. Addressing revisions to the Brief Fear of Negative Evaluation scale: measuring fear of negative evaluation across anxiety and mood disorders. J Anxiety Disord 2011 Aug;25(6):822-828. [CrossRef] [Medline]
- Radloff LS. The CES-D scale: a self-report depression scale for research in the general population. Appl Psychol Meas 1977;1(3):385-401. [CrossRef]
- Fat LN, Scholes S, Boniface S, Mindell J, Stewart-Brown S. Evaluating and establishing national norms for mental wellbeing using the short Warwick-Edinburgh Mental Well-being Scale (SWEMWBS): findings from the Health Survey for England. Qual Life Res 2017 May;26(5):1129-1144 [FREE Full text] [CrossRef] [Medline]
- Maheswaran H, Weich S, Powell J, Stewart-Brown S. Evaluating the responsiveness of the Warwick Edinburgh Mental Well-Being Scale (WEMWBS): group and individual level analysis. Health Qual Life Outcomes 2012 Dec 27;10:156 [FREE Full text] [CrossRef] [Medline]
- Ware JE. In: Maruish ME, editor. The Use of Psychological Testing for Treatment Planning and Outcomes Assessment. New York: Routledge; 1999.
- Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999 Sep;89(9):1322-1327. [CrossRef] [Medline]
- Angrist JD, Imbens GW, Rubin DB. Identification of causal effects using instrumental variables. J Am Stat Assoc 1996;91(434):444-455. [CrossRef]
- White IR, Carpenter J, Evans S, Schroter S. Eliciting and using expert opinions about dropout bias in randomized controlled trials. Clin Trials 2007;4(2):125-139. [CrossRef] [Medline]
- Wallace P, Murray E, McCambridge J, Khadjesari Z, White IR, Thompson SG, et al. On-line randomized controlled trial of an internet based psychologically enhanced intervention for people with hazardous alcohol consumption. PLoS One 2011 Mar 9;6(3):e14740 [FREE Full text] [CrossRef] [Medline]
- StataCorp. College Station, Texas: StataCorp LLC; 2017. Stata Statistical Software: Release 15 URL: https://plattogvibinta.cf/stata-statistical-software-release-15.html
- Curtis L, Burns A. Unit Costs of Health & Social Care 2017. University of Kent, Canterbury, UK: Personal Social Services Research Unit (PSSRU); 2017.
- Sheffield Licensing - University of Sheffield. Sheffield, UK: University of Sheffield SF-6D: Calculating QALYs from the SF-36 and SF-12. URL: https://licensing.sheffield.ac.uk/i/health-outcomes/SF-6D.html [accessed 2019-10-01]
- Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data. JAMA Psychiatry 2017 Apr 1;74(4):351-359. [CrossRef] [Medline]
- Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on Internet-based mental health interventions — a systematic review. Internet Interv 2014;1(4):205-215. [CrossRef]
- Mathieu E, McGeechan K, Barratt A, Herbert R. Internet-based randomized controlled trials: a systematic review. J Am Med Inform Assoc 2013 May 1;20(3):568-576 [FREE Full text] [CrossRef] [Medline]
- Murray E, White IR, Varagunam M, Godfrey C, Khadjesari Z, McCambridge J. Attrition revisited: adherence and retention in a web-based alcohol trial. J Med Internet Res 2013 Aug 30;15(8):e162 [FREE Full text] [CrossRef] [Medline]
- Donker T, Griffiths KM, Cuijpers P, Christensen H. Psychoeducation for depression, anxiety and psychological distress: a meta-analysis. BMC Med 2009 Dec 16;7:79 [FREE Full text] [CrossRef] [Medline]
- Thompson AE, Anisimowicz Y, Miedema B, Hogg W, Wodchis WP, Aubrey-Bassler K. The influence of gender and other patient characteristics on health care-seeking behaviour: a QUALICOPC study. BMC Fam Pract 2016 Mar 31;17:38 [FREE Full text] [CrossRef] [Medline]
- Boettcher J, Renneberg B, Berger T. Patient expectations in internet-based self-help for social anxiety. Cogn Behav Ther 2013;42(3):203-214. [CrossRef] [Medline]
- Nordgreen T, Havik OE, Ost LG, Furmark T, Carlbring P, Andersson G. Outcome predictors in guided and unguided self-help for social anxiety disorder. Behav Res Ther 2012 Jan;50(1):13-21. [CrossRef] [Medline]
|ANU: Australian National University|
|BFNE-S: Brief Fear of Negative Evaluation|
|CACE: Complier Average Causal Effect|
|NHS: National Health Service|
|QALY: quality-adjusted life year|
|SPIN-17: 17-item Social Phobia Inventory|
|SWEMWBS: Short Warwick-Edinburgh Mental Well-Being Scale|
Edited by G Eysenbach; submitted 26.10.19; peer-reviewed by N Van-Zalk, C Bedard; comments to author 16.11.19; revised version received 22.11.19; accepted 10.12.19; published 10.01.20
©John Powell, Veronika Williams, Helen Atherton, Kylie Bennett, Yaling Yang, Mina Davoudianfar, Annika Hellsing, Angela Martin, Jill Mollison, Milensu Shanyinde, Ly-Mee Yu, Kathleen M Griffiths. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 10.01.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.