An RCT compared the efficacy of the fully automated, stand-alone Thrive intervention to a waitlist control (WLC) in reducing depression symptoms among adults. Participants were recruited from communities across Montana and immediately randomized 1:1 to the intervention group (Thrive) or a WLC group (delayed access to Thrive until the 8-week follow-up assessment) after meeting the inclusion criteria and providing electronic informed consent on the study website [39] (Multimedia Appendix 1). Participants in both groups were assessed for primary and secondary outcomes at baseline and 4 and 8 weeks postenrollment. Enrollment occurred between September 2017 and January 2018, and all assessments were completed by March 2018. The Montana State University Institutional Review Board approved the protocol and all related materials (#MS033017-FC) prior to study initiation. The study is registered at ClinicalTrials.gov (NCT03244878).

Participants were recruited using community fliers, public service announcements, local newspaper advertisements, newsletters, and community social media sites. The study was also promoted through select state organization email listservs, large employers, local health care providers, Facebook, a Craigslist community page, community meeting events, and Montana State University Extension faculty communications with their respective counties. All methods directed potential participants to a study website, which informed potential applicants about the study expectations; determined their eligibility; and guided those eligible through the informed consent, randomization, and assessment process [39].

Inclusion criteria included adults aged ≥18 years with mild-to-severe depression severity (PHQ-9 score >5) [37]; Montana residency; a valid email address; and regular access to broadband internet via a computer, tablet, or smartphone. At enrollment, potential participants who indicated recent suicidal ideation (PHQ-9 item 9 score >0) were asked to confirm that they could stay safe and those responding that they could not were considered ineligible (see the safety protocol description below for the handling of cases of suicidal ideation and Multimedia Appendix 4). All participants provided electronic informed consent prior to study participation. All participants were informed that they were free to obtain and use any additional care available throughout their participation in the trial. In total, 463 participants were enrolled, and of these, 109 were deemed fraudulent identities and their data were removed prior to analyses (Multimedia Appendix 2). Additionally, three individuals provided invalid email addresses, which led to their exclusion from study participation, and data from two control participants were discarded, as they were accidentally provided access to the intervention immediately. Six participants were excluded because of missing baseline data on the covariate “currently receiving psychosocial therapy for depression,” which was required for data analysis. The final analytic sample included 343 participants (intervention: n=181; control: n=162), of which 86 (25%) participants did not complete any follow-up assessments (Multimedia Appendices 2 and 3).

Participants completed study assessments on the study website at baseline, 4 weeks, and 8 weeks. Automated email reminders were sent when each assessment was due, with two additional follow-up automated reminder emails sent within 7 days to those who had not completed the assessment. If the participant did not complete the scheduled assessment within a 10-day window, data were considered lost to follow-up for that assessment (Multimedia Appendix 10). A US $25 Amazon gift code was sent following completion of each assessment.

The primary outcome measure was participants’ self-reported depression symptom severity PHQ-9 score. Prespecified secondary outcomes included anxiety symptom severity, daily functioning, and resilience. Suicidal ideation was added as a secondary outcome after the original protocol was developed, but prior to the start of analyses.

All outcomes were assessed using self-reported, validated measures automatically administered via the study website. Depression symptom severity was evaluated using the PHQ-9, which incorporates the major depressive disorder “A criterion” symptoms of Diagnostic and Statistical Manual of Mental Disorders (DSM)-IV and DSM-5 (score range: 0-27, with higher scores indicating worse symptoms) [37]. Anxiety symptom severity was measured with the Generalized Anxiety Disorder Scale (GAD-7, score range: 0-21), with higher scores indicating worse symptoms [44]. Daily functioning was measured using the Work and Social Adjustment Scale (WSAS, score range: 0-40), with higher scores indicating a greater impact of depression on daily functioning [45]. The abbreviated version of the Conner-Davidson Resilience Scale (CD-RISC-10) measured resilience (score range: 0-40), with higher scores indicating greater resilience [46]. Lastly, suicidal ideation was assessed using item 9 of the PHQ-9 measure (“Thoughts that you would be better off dead or of hurting yourself” in the past two weeks). Item 9 was treated as an ordinal scale that ranged from 0 (“not at all,” no suicidal ideation) to 3 (“nearly every day”) [37].

When study applicants endorsed having at least some suicidal ideation (PHQ-9 item 9 score>0) at study enrollment, the study website displayed multiple sources of immediate help (Multimedia Appendix 4) and asked applicants to declare whether they were sure they could stay safe. Applicants answering that question negatively were considered ineligible to participate in the study. Notably, no applicants reported that they could not stay safe. Regardless of the response to the “stay safe” question, the study website prompted the individual to seek help and provided the same sources of help described in Multimedia Appendix 4. If an enrolled participant indicated s/he had at least some suicidal ideation during the 4- or 8-week assessments (PHQ-9 item 9 score >0), the study website immediately prompted the individual to seek help and provided the same sources of help. Additionally, the Thrive intervention directed individuals with PHQ-9 scores ≥20 at any assessment and those with PHQ-9 scores ≥10 on the third assessment to seek help from a doctor. A thorough description of all safety measured used is described in Multimedia Appendix 4. Participants were provided email addresses (lead investigator and IRB Chair) for reporting of any adverse experiences or events.

An initial pool of 10 variables was selected a priori for analysis as potential covariates of depression severity, anxiety, functional impairment, resilience, and suicidal ideation (Multimedia Appendix 5).

Outcomes were assessed at baseline and at 4 and 8 weeks. The change over time in each continuous outcome and suicidal ideation was compared between the intervention and control groups using a linear mixed model analysis of repeated measures and an ordinal logistic regression model within a Generalized Estimating Equation framework, respectively. A separate model was conducted on each outcome measure. Each model contained fixed-effects terms for treatment (intervention vs control), time, treatment × time interaction, and respective baseline measure (prior to the intervention) as covariates. Receiving therapy for depression (yes/no) at the baseline assessment was also included as a covariate in each model. Least squares means (adjusted treatment means) and adjusted odds ratios (OR) were estimated as part of the mixed model and ordinal logistic model, respectively, to interpret the treatment effect. For the ordinal logistic regression, the cumulative probabilities were modeled over the higher-ordered suicidal ideation scale scores (more suicidal ideation).

Statistical analyses were performed using SAS software, version 9.4 (SAS Institute, Inc, Cary, North Carolina). Maximum likelihood estimators allow efficient parameter estimation using only available data under an assumption of missing at random [47-49]. The level of significance was set at α=0.05 (two-tailed), and the Bonferroni method was implemented to control false-positives over the multiple tests. The 95% CIs for the point estimates of treatment group effects were also adjusted to match the Bonferroni adjustment to significance levels in the corresponding test. A priori evaluable sample size for a statistical power of 80% was estimated (Multimedia Appendix 6).

Participant baseline characteristics are described in Table 1. The cohort was predominately female (290/343, 85%) and Caucasian (319/343, 93%), with a mean age of 42.9 (SD 13.3) years. Most participants were married (198/343, 58%); employed, or in school (265/343, 77%); had some higher education (318/343, 93%); and lived in rural regions (287/343, 84%). In addition, 59% (202/343) of participants were receiving medication, psychosocial therapy, or other treatment for depression or anxiety at baseline. No significant differences for any outcome measure were observed at baseline (Table 2) nor did the baseline depression symptom severity differ between groups (Multimedia Appendix 7). Participants randomized to the intervention group completed a mean of 8.7 lessons. No adverse experiences or adverse events were reported by either study group.

This study evaluated the effectiveness of a fully automated, stand-alone, video-centric iCBT intervention (Thrive) for reducing depression symptom severity among adults residing in Montana. The design incorporated features of practical, pragmatic, and community-based effectiveness trials: liberal inclusion criteria (mild to severe depression symptoms), allowance for any past and concurrent treatments, and broad (symptom, functional, and resilience) outcomes. Over 8 weeks, depression symptom severity (primary outcome) in the Thrive intervention group was significantly reduced as compared to the control group. Additionally, the Thrive intervention group showed significant improvements in anxiety symptoms, work and social functioning, and resilience compared with the control group.

Although the Thrive intervention, like other iCBT interventions for depression in non-US populations [19], may be equally effective in rural and urban populations, the intervention has several characteristics that may make it appealing to rural US residents as an alternative or supplemental mental health intervention for depression symptoms. A previous qualitative study of Thrive among rural Montana adults led to an adaptation of 6 of the >300 videos to better reflect scenarios and settings common in rural communities [41], which may improve user engagement. Thrive can be performed in the privacy of one’s home, which might offset mental health-related stigma, a frequent barrier to seeking mental health services experienced by rural US residents [51]. Thrive is accessible from any location with internet access and can be used on smartphones, tablets, and computers, making it an easily accessible, cost-effective alternative for rural residents who often have to drive long distances for in-person mental health care [41].

To our knowledge, the clinical effect of the Thrive intervention on depression symptoms is one of the largest reported for an iCBT intervention delivered without clinician interaction. Omitting clinician support allows for broader generalizability to community populations by limiting costs and avoiding barriers associated with an underresourced mental health workforce. A recent meta-analysis of 16 self-guided iCBT studies by Karyotaki et al [21] and a systematic review by Lorenzo-Luaces et al [22] of the same 16 and an additional 5 self-guided iCBT studies reported that 7 studies evaluated iCBT interventions that, like Thrive, had no clinician contact [25-31]. The impact of Thrive on depression symptoms (d=0.63) was greater than or similar to the 7 comparable studies (nonsignificant [25]; d=0.17 [26]; d=0.20 [28]; d=0.28 [27]; d=0.30 and 0.65 [for analysis of variance and mixed model analyses, respectively] [29]; g=0.36 [31]; and d=0.50 [30]). The greater impact of Thrive may be a result of the intervention group completing more lessons on average (8.7) than some other fully automated, stand-alone iCBT interventions [24-26].

Compared to self-guided iCBT interventions delivered with meaningful clinician support, described in Karyotaki et al [21] and Lorenzo-Luaces et al [22], the Thrive intervention yielded a similar or greater effect size for depression symptoms: Four studies that used clinician/research personnel contact to determine eligibility found effect sizes of d=0.08 [52], d=0.38 [53], d=0.8 [54], and g=0.76 for depression symptoms [55], although the latter study excluded “outliers” from the primary analysis and stated their inclusion resulted in “nonsignificant findings” and reduced the effect size by an undisclosed amount [55]. Two other studies that used iCBT in combination with contact with a therapist also reported similar or lower effect sizes (d=0.51 [56] and 0.38 [31]) than those reported for Thrive. Six studies that included individualized email or SMS check-in reminders or telephone diagnostic interviews produced nonsignificant [57] or small-to-medium effect sizes for depression symptoms (d=0.34 [30]; d=0.36 [58]; d=0.55 [59]; d=0.57 [60]; and d=0.66 [61]). Finally, the six studies providing up to weekly nonautomated email or telephone support reported no significant differences compared to the control for two studies [24,62] and small-to-large significant effects sizes for four studies: (1) d=0.22 and d=0.34 for support upon request and weekly support, respectively [26]; (2) d=0.4 (pre-post within-subjects effect size) [63]; (3) d=0.39 [64]; and (4) d=1.14 [61].

This study found similar or greater effect sizes for secondary outcomes (anxiety symptoms [d=0.47], work and social functioning [d=0.39], and resilience [d=0.55]) compared to the 21 iCBT trials described in the recent meta-analysis and systematic review [21,22]. In the seven studies reporting anxiety symptom outcomes, four identified nonsignificant differences with the control [53,56,60,62] and three found small-to-nearly moderate effect sizes (d=0.22 [54], d=0.25 [26], and d=0.43 [30]). The three studies that assessed life/work functioning (using the same measure reported here [WSAS]) found a nonsignificant difference as compared to the control [62], a significant improvement for iCBT+treatment as usual but not in iCBT compared to the control (effect sizes not provided) [52], and an effect size of 0.36 [29]. Two other studies found no significant difference [57] and an effect size of 0.65 [54] when measuring life/work functioning with the Sheehan Disability Scale. To our knowledge, no other iCBT studies have evaluated resilience. Lastly, this study found a nonsignificant reduction (58%) in the odds of having greater suicidal ideation in the Thrive intervention group than in the control group at 8 weeks. A small effect size (d=0.20) was found in the single prior study that measured the impact of an iCBT intervention on suicidality [58].

The design of this study supports the generalizability of its findings in community settings. First, the minimal eligibility restrictions indicate the potential applicability of the Thrive intervention in general populations outside the controlled research setting. Of the seven studies described in Karyotaki et al [21] and Lorenzo-Luaces et al [22], only one that delivered a fully automated, stand-alone iCBT intervention like the Thrive intervention had similarly broad eligibility criteria [29]. Second, omitting clinician-administered diagnostic evaluations, as used in several iCBT studies [57,59-61], allows for broader generalizability to community settings. The use of person-administered diagnostics greatly increases costs, limits broad intervention dissemination due to an insufficient mental health workforce, and decreases the potential public health impact of fully automated, stand-alone iCBT interventions.

This study had several limitations. Like most iCBT studies [21], assessment completion rates were low, with 68% and 65% of randomized subjects completing assessments at 4 and 8 weeks, respectively. These rates are slightly below the 73% (range 55%-95%) unweighted mean completion rate [24-30,52,55,58-64] for short-term (6-16 weeks) follow-up assessments of all studies in Karyotaki et al [21]. Additionally, the study offered monetary incentives for 4- and 8-week survey completion, which are associated with greater response in electronic questionnaires [65] and retention rates [66], which limits conclusions about uptake of Thrive in nonresearch settings. Monetary incentives have been used in all six RCTs of iCBT interventions for depression in US populations [24,25,27,28,54,57] and three of the seven (43%) RCTs of fully automated, stand-alone iCBT interventions reported in Karyotaki et al and Lorenzo-Luaces et al [21,22,25,27,28]. Although using self-assessments, a common practice in iCBT studies, is a potential weakness of the study, the use of validated, widely used instruments largely addressed this issue. The PHQ-9 (depression symptoms), Generalized Anxiety Disorders Scale-7 (anxiety symptoms), and WSAS-5 (functioning) correlate well with clinician-administered instruments [37,67,68] and are sensitive to interventional effects [45,69,70]. Of note, self-assessments may underestimate the effect of iCBT relative to similar studies using clinician-administered assessments [71]. The study eligibility, which omitted the need for a depressive disorder diagnosis and included participants across the full range of depression symptom severity increases the study’s generalizability. However, the make-up of the recruited study population, with high a proportion of female (85%) and white (93%) participants with at least some education after high school (69%) limits its generalizability. Greater participation of women is a common limitation among studies of iCBT for depression. For example, Karyotaki et al [21] reported that 66% of all participants were female, and among the studies of fully automated, stand-alone iCBT interventions, female participation ranged from 65% to 81% [21]. The high proportion of female participants across iCBT studies could be due, in part, to the greater prevalence of depression in women [2] or as a yet-unidentified barrier for men. The relatively high percentage (23.5%) of fraudulent participants identified is another limitation, which is becoming increasingly common in internet-based studies [72]. The number of fraudulent participants in this study is similar to that in several other studies using Web-based surveys: 28.7% [73], 20.5% [74], and 18.7% [75]. Although the study was powered appropriately to detect between-group differences in a priori defined primary and secondary outcomes, even when allowing for corrections for multiple testing (design strength of this study), it was underpowered to detect meaningful difference in the post-hoc assessment of suicidal ideation, as only 41% of subjects experienced suicidality at baseline. In addition, the 8-week evaluation period cannot inform on the long-term impact of the Thrive intervention when delivered in community settings. Longitudinal studies with longer follow-up periods are warranted to determine whether iCBT programs impact the risk of depression relapse [10]. Finally, as a community-based trial, these findings cannot be generalized to clinical settings.

Thrive, a fully automated, stand-alone iCBT intervention, demonstrated greater short-term improvements in depression and anxiety symptoms, work and social functioning, and resilience compared to control participants in a broad community population with mild to severe depression symptoms. A trend toward greater reduction in suicidal ideation was also observed. The magnitude of clinical benefit seen with the Thrive intervention appears to be greater than most other fully automated, stand-alone interventions and similar to those requiring clinician support. Thus, Thrive represents a potentially effective and cost-effective solution for treating depression symptoms in rural regions that often have few or no mental health providers. Evaluations of the impact of the Thrive intervention over longer periods of time and other population settings are warranted.