Depressive disorders are among the most frequently occurring mental disorders, with a prevalence rate of 10.7% within a 12-month period [1]. They affect >280 million people worldwide [2] and are ranked fourth in terms of disease burden [3]. At the individual level, they have adverse effects on one’s cognitive functioning, quality of life, mortality, and other health outcomes [4-6]. At the macro level, they have tremendous economic costs worldwide [7,8].

Psychological interventions, especially cognitive behavioral therapy (CBT), have demonstrated to be efficacious in treating depressive disorders [9-12]. They are not only as efficacious as antidepressant medications [13], but they are also preferred by people with depression [14]. Despite its demonstrated efficacy, the uptake of psychotherapy has been poor, with only 13.8% of those who experience mood disorders receiving psychological interventions [15]. With the emergence of digital-based psychological interventions (DPIs), the uptake of evidence-based psychotherapy may be expedited through access over the internet.

A psychological intervention is considered digital when technology is used in its delivery [16]. Although varying in forms, DPIs typically use a software program, website, or app to disseminate therapeutic contents displayed in texts, audios, or videos [17]. Although most modern DPIs run on the internet, offline interventions may also be considered digital whenever technology is involved, for example, a therapy program installed on a web-free computer or that runs on a CD-ROM [18]. Nonetheless, some DPIs would also come with human support, for instance, having clinicians to review homework assignments and provide feedback on users’ progress [19].

DPIs have overcome major help-seeking barriers by reducing time and costs [20] and mitigating stigma through anonymity and privacy [21]. DPIs’ effectiveness in treating common mental health conditions was found to be comparable with that of face-to-face psychological interventions [22,23]. Previous meta-analytic studies [24-27] have demonstrated their efficacy in alleviating depressive symptoms among adults when compared with a control group, with between-group effect sizes ranging from g=−0.41 to g=−0.90.

Changes in depression during treatment can be attributed to at least 5 distinct sources of effects [28]. These include the treatment-specific effect, which is the net effectiveness of a treatment; the nonspecific treatment effects such as attention, support, and expectations of being treated; spontaneous remission; regression to the mean; and other treatment-unrelated factors. Unlike pharmacological trials, where blinding is relatively easy to carry out, participants in psychological intervention trials know immediately the results of assignment upon allocation. Any changes in the outcome can be regarded as a blend of true treatment effects and an array of other non–treatment-specific effects. In other words, a proportion of changes experienced by the clients or users during the therapy process may come from sources other than the designated treatment.

Ideally, randomized controlled trials (RCTs) that use adequate control conditions should have controlled for these nontreatment effects [29]. The established evidence we have today is largely generated by RCTs and has been consolidated in meta-analyses that focused on between-group effects comparing an intervention with a control group. In a recent meta-analysis, Moshe et al [27] found that the types of control conditions significantly moderated the effect sizes of DPIs, implying a therapeutic potential in these control conditions.

Mohr et al [30] conducted a comprehensive review of the control conditions in RCTs examining the efficacy of psychological treatments for depression. They found that the choice of control condition can have a very large impact on the outcome of a study. In general, studies that used a waitlist or treatment-as-usual control produced the largest between-group effect favoring the examined treatment, whereas those with no-treatment or pill or psychological placebo controls found moderate effects in active treatment conditions, and those with active controls found the least effects in active treatment conditions.

Cuijpers et al [31] attempted to estimate the effects of nontreatment factors in a meta-analysis by investigating the effects of face-to-face nondirective supportive therapy. They found that nearly one-third of the observed changes in depression could be attributed to spontaneous remission and nearly half of the contribution could be accounted for by nonspecific factors. Subsequently, in their meta-analysis investigating remission in people receiving face-to-face psychological treatment, they noted an 18% reduction in depression (Beck Depression Inventory-II) scores among people in the control conditions [32]. More recently, in light of the high heterogeneity of the commonly used care-as-usual (CAU) controls, Cuijpers et al [33] investigated the different categories of CAU and found that the effects of psychotherapies did not differ significantly across CAU variants.

Another meta-analytic study conducted by Whiteford et al [34] examined the probability of remission from untreated depression across 19 studies and reported a high rate of spontaneous remission (53% within 12 months) among participants in waitlists and primary care settings. It was noteworthy that Furukawa et al [35] compared waitlist, no-treatment, and psychological placebo controls with face-to-face CBT and also found that different control conditions lead to substantial but different treatment effect estimates. The waitlist was regarded by the authors as a “nocebo” condition given its inferiority as compared with the no-treatment controls, in which the participants do not expect to receive any active treatment after the study period is over.

The aforementioned findings indicate a potential moderating effect of the control treatment on the reported effect sizes of face-to-face psychotherapies. Given that investigations have focused primarily on face-to-face psychotherapies, how control conditions perform in general in RCTs of DPIs has not been systematically investigated. In the past decade, the use of DPIs for health care has become a burgeoning area of investigation. A massive demand on the use of DPIs has been particularly evident since the COVID-19 pandemic lockdowns, when face-to-face services were stalled [36].

Participants who joined face-to-face psychotherapies might have had different demographic profiles and preferences from those who were willing to receive DPIs. The latest research on the digital divide has shown that digital use is greatly influenced by sociodemographic factors and lifestyle differences [37]. Thus, participants in DPI studies may represent a heterogeneous subgroup of individuals different from those who opt for face-to-face therapies. Updated insights into the characteristics of these individuals are warranted. Nevertheless, an in-depth understanding of how control conditions in RCTs of DPIs fare is necessary for researchers, practitioners, and consumers to evaluate the current evidence on DPIs.

This systematic review and meta-analysis aimed to investigate the possible effects of common control comparators in studies on DPIs for depression among adults. The primary objective of this meta-analytic review was to provide answers to the following research questions: (1) What are the sizes of the within-subject effects of the control conditions over time? (2) How do different types of control conditions differ in terms of time effects? and (3) How does baseline severity affect the change trajectory in control condition participants over time?

This meta-analysis included studies that fulfilled the following criteria pertaining to participants, intervention, and publication type.

A systematic literature search was carried out on September 28, 2021, on selected electronic databases: PubMed; ProQuest; Web of Science; and the Ovid system with MEDLINE, PsycINFO, and Embase. Details of the search strategy are presented in Multimedia Appendix 1. This search strategy resulted in a total of 9249 titles and abstracts across the different databases. All search results were entered into Zotero (Corporation for Digital Scholarship), a reference manager for review and removal of duplicate citations [39]. After the removal of duplicates, a total of 49.41% (4570/9249) of the records were retained.

A total of 3 researchers (AT, FH, and OC) then screened the titles and abstracts to eliminate records that obviously did not meet the inclusion criteria (eg, studies with children or older persons, not related to depression, and not on DPIs). Excellent interrater reliability was established (intraclass correlation coefficient=0.964) between the 3 reviewers based on the first 2.19% (100/4570) of screened articles.

A total of 256 records that potentially met the inclusion criteria were identified, and the reviewers then read the full texts of these records to confirm their eligibility. Eventually, 41.8% (107/256) of these studies were left for data extraction. Figure 1 presents the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart of the selection process.

Participant characteristics in each study, including age, the percentage of female participants, and tertiary education, were recorded. The percentage of participants taking antidepressant medications at the time of the study and previous exposure to psychotherapy were also extracted.

In addition to the therapeutic approach (eg, CBT, psychodynamic therapy, or positive psychology) of the comparator intervention (in the intervention group), information related to the control group, including types (see detailed descriptions in the Results section), a description of the intervention received by the control participants, and the mode of delivery, was also extracted. In the event of studies having more than one control group (eg, a waitlist and an active control), we extracted both conditions as separate data as the results for these subsamples were considered to reflect the outcomes of 2 studies. The time passed (in weeks) since the baseline assessment and the retention rate among the controlled participants at each assessment time point were also recorded.

The primary outcome of interest was the within-subject change in depressive symptoms over time. Wherever available, the observed means and SDs of depressive symptoms at baseline, postintervention measurement, and follow-ups were extracted. SEs were converted into SDs by multiplying the square root of the sample size. When the aforementioned information was not available, we checked whether effect sizes with CIs and mean changes with SDs were available. In cases where more than one depressive symptom measure was used, all of them were extracted and aggregated in the meta-analysis.

Study quality was assessed using the Cochrane risk-of-bias tool for randomized trials version 2 (RoB 2) [40]. The RoB 2 is a structured tool that assesses possible sources of bias in a trial across five different domains: (1) the randomization process, (2) deviation from the intended interventions, (3) missing outcome data, (4) measurement of the outcomes, and (5) selection of the reported results. A high-quality study should be at low risk in all 5 domains. In total, 2 reviewers (FH and OC) independently assessed the studies based on the RoB 2, and a third reviewer (AT) made the final decision whenever there was disagreement between the reviewers.

Meta-analyses were performed using Comprehensive Meta-Analysis (version 3; Biostat, Inc) [41] to determine the mean differences in depressive symptoms across different time durations (in weeks) since baseline. The duration was divided into four intervals: (1) immediate (≤8 weeks), (2) short term (9-24 weeks), (3) medium term (25-48 weeks), and (4) long term (>48 weeks). These time cutoffs were chosen based on the distribution of the assessment time points throughout the selected studies. In cases where more than one assessment time point fell within the same interval (eg, 18 and 24 weeks), we used the one closer to the far end of that interval (ie, 24 weeks).

The correlation between pre-post scores on the outcome measures was assumed to be r=0.59 with reference to Balk et al [42]. Effect sizes (Hedges g) were calculated using the standardized mean difference (SMD) approach. The interpretation of effect size follows Cohen [43], where 0.2 represents a small effect, 0.5 represents a moderate effect, and 0.8 represents a large effect. The extent of heterogeneity between the studies was assessed using the I² statistic, where a value of ≥75% represents significant heterogeneity. Random-effects models were calculated assuming significant heterogeneity [44].

Publication bias was examined using funnel plot inspection and Egger regression [45]. Symmetry of the funnel plot and a nonsignificant Egger regression indicated an absence of publication bias. Whenever publication bias was detected, the trim-and-fill procedure by Duval and Tweedie [46] was used to obtain a sensitivity estimate of the effect size.

Apart from the differences in effect sizes at various time intervals, subgroup analyses were performed to explore the differential impacts of control group types on severity of depressive symptoms in effect sizes. Only subgroups that comprised ≥2 studies were interpreted to ensure the stability of the effect size estimates.

Depressive symptom severity was categorized into 3 levels—mild, moderate, and severe—based on the proposed cutoffs provided in the respective publications (see Multimedia Appendix 1 for the cutoffs). Finally, we also explored the meta-regression effects of age, sex ratio, and higher education as an exploratory analysis in explaining the heterogeneity across the studies.

A total of 107 peer-reviewed RCTs meeting the inclusion criteria with an aggregation of 11,803 control participants were included in this meta-analysis. The studies were conducted between 2003 and 2021. The years with the most publications were 2020 and 2021, each with 13.1% (14/107) of the studies included. More than half of them (60/107, 56.1%) were conducted in Europe (eg, the Netherlands, Germany, and Sweden) and the United Kingdom. Australia (17/107, 15.9%) and the United States (18/107, 16.8%) also contributed substantially. Other regions included China, Singapore, Japan, Nigeria, Oman, and Colombia. Most of these studies recruited participants from the community (52/107, 48.6%) and in clinics (36/107, 33.6%).

The reported mean age of the participants was 38.48 (SD 7.69) years, and the mean percentage of female participants was 71.54% (SD 15.38%). Approximately half of the participants (mean 54.1%, SD 20.66%) attained higher education, as defined by having finished college or university, and were either married or living with partners (mean 52.26%, SD 19.98%), and 68.19% (SD 19.12%) were employed (including being a student) at the time of the study. Although not reported in many studies, the mean percentage of participants taking antidepressant medication was 44.86% (SD 23.59%), and 51.55% (SD 22.41%) had experience receiving psychotherapy before the study, as reported in 52.3% (56/107) and 38.3% (41/107) of the studies, respectively. The most frequently used assessments of depressive symptoms were the Patient Health Questionnaire-9 (43/107, 40.2%) and the Beck Depression Inventory-II (40/107, 37.4%). Key features of the 107 included studies are summarized in Table 1.

The overall quality of the included studies was not optimal according to the assessed results of the RoB 2 (Multimedia Appendix 2 [47-153]). Of the 107 publications, 11 (10.3%) were judged as having a high risk of bias. Most studies (91/107, 85%) were judged as having “some concerns,” as defined by the RoB 2. Most of these concerns were related to the lack of blinding, either to the participants or to the assessor, and therefore, this posed some concerns over the risk of bias associated with the measurement of the outcome. Only 4.7% (5/107) of the studies were considered high-quality.

A total of 113 control arms (some studies included multiple control arms) were coded in the included studies. In total, 6 distinct types were identified based on the contents and delivery modes stated in the articles.

The mean time interval was 8.88 (SD 6.06) weeks between baseline and the postassessment time point, 21.87 (SD 12.5) weeks between baseline and the first follow-up, and 41.38 (SD 30.31) weeks between baseline and the second follow-up. The mean retention rate was 79.54% (SD 17.05%) at the postassessment time point, 75.13% (SD 17.56%) at the first follow-up, and 66.76% (SD 16.90%) at the second follow-up. The retention rates did not differ significantly across control types at the postassessment time point (F_5,92=0.8; P=.53) and at the first (F_5,48=0.7; P=.63) and second (F_5,14=0.8; P=.57) follow-ups.

Considering the differential effect of control type, the difference between subgroups was significant in the immediate (Q_between=10.48; P=.03) and medium term (Q_between=7.85; P=.02) but not in the short term (Q_between=10.06; P=.07), and there were insufficient data to compare in the long term. As illustrated in Figure 3, the CAU control type (thick dashed line and circle markers) produced the strongest effect overall after 8 weeks, reaching its peak of g=−0.950 (95% CI −1.161 to −0.739) in the medium term. The waitlist control (thick solid line and diamond markers) produced relatively smaller effects as compared with the other control types.

The effect of active controls (thin solid line and star markers) was small shortly after the baseline assessment (g=−0.207) but strengthened to a moderate effect in the medium term (g=−0.653). Sham controls (thin dotted line and “+” markers) consistently produced close-to-moderate effects from the immediate term (g=−0.462) to the short term (g=−0.614).

Although most control types have accumulating effects over time, information controls (thin solid line and cross markers) showed a drop from a moderate effect (g=−0.585) to a small effect (g=−0.407) in the medium term. Finally, only 3.7% (4/107) of the studies (n=192 participants) used a monitoring control type (triangle marker), which was assessed in the short term in all cases. The pooled effect of these monitoring controls was moderate (g=−0.519).

Considering the differential effect of severity, subgroup analysis results showed significant differences across levels of severity in the immediate (Q_between=19.65; P<.001), short (Q_between=9.02; P=.01), and medium (Q_between=7.69; P=.02) term but not in the long term (Q_between=1.18; P=.28). As illustrated in Figure 4, studies in which there were lower levels of severity at baseline appeared to produce smaller effects in reducing depressive symptoms in the immediate, short, and long term but not in the medium term.

Depressive symptoms did not change significantly among studies in which there were mild levels of baseline symptoms in the immediate (P=.14) and short (P=.05) term. Studies in which there were moderate levels of depressive symptoms were estimated to produce the largest effect on symptom reduction in the medium term (g=−0.894). In addition, an opposite trend between studies in which there were moderate (solid line in Figure 4) and severe (dotted line) levels of symptoms in the medium and long term was observed. Specifically, the effect started to shrink in the moderate-severity subgroup but started to grow in the severe subgroup beyond 48 weeks.

As the heterogeneity was very high (>75%) even after subgrouping, a series of meta-regression analyses were performed to examine whether participants’ age, gender, and education level accounted for some of the heterogeneity. As shown in Table 3, in terms of unique variance, female participants (z=−2.37; P=.02) and education level (z=1.98; P=.047) in the short term as well as age (z=2.15; P=.03) in the medium term appeared to significantly explain some variance in the effect sizes. Specifically, the coefficients indicated that the higher the proportion of female participants and the lower the proportion of participants who had a higher education in a study, the smaller the effect in the short term; studies with older participants were found to obtain larger effect sizes in the medium term. However, when considering all covariates collectively, none of the regression models were significant, indicating that the moderating effect of the covariates was very limited and that these covariates together were unable to explain the variance in effect sizes at each of the time intervals.

This meta-analytic study examined the magnitude of the effects associated with different types of control conditions in DPI trials targeting depressive symptoms among adults. On the basis of the 107 included studies, we observed a U-shaped trend in depressive symptom reduction over time. Specifically, the effect sizes were small to moderate (g=−0.358) in the first 8 weeks since the baseline assessment; then, a moderate effect (g=−0.549) was noted between the 9th and 24th weeks, which was approximately half a year later. After adjusting for publication bias, this short-term effect was reduced to a small to moderate effect (g=−0.404). Within the second half of the same year (25th to 48th weeks), the effect peaked at g=−0.810 (adjusted: g=−0.573), indicating a large (moderate after adjustment) effect. Although the effect shrank slightly beyond 48 weeks (a year later), it was maintained at a close-to-large effect range (g=−0.769).

To the best of our knowledge, this study is the first attempt that looked at the longitudinal trajectory of depressive symptoms among control participants in DPI studies. Undoubtedly, the effect sizes obtained in the control conditions were generally smaller than those of the intervention conditions that were reported [22-27]. However, the findings concur with those of previous studies on face-to-face interventions that, even without active interventions, symptomatic remission is possible [31,33,34]. Considering people in the waitlist control group who were highly restricted on the receiving of treatment procedures, the reduction in depressive symptoms reflected a collective variance explained by spontaneous recovery and regression to the mean, at least to some extent. Although this subtype of control yielded relatively weaker effects than other control conditions given its passivity, the improvement was still significant and meaningful.

Regarding the suspicion of Furukawa et al [35] that people being assigned to a waitlist control may hinder their self-healing behaviors and, thus, experience a nocebo effect (ie, worsening of depression while waiting), our findings did not support this claim as waitlist control groups in our meta-analysis reduced depressive symptoms significantly within the first 24 weeks. However, given that the authors separated waitlist from no-treatment control groups, whereas we did not identify a purely no-treatment condition, our findings could not refute the nocebo hypothesis entirely. The authors of that study investigated face-to-face CBT; it may also be possible that the nocebo effect only applies to people who are willing to receive face-to-face psychotherapy but not to those receiving DPIs.

Moreover, the results concerning CAU were difficult to interpret given the broad definition of “usual treatment” used. Of the 44 studies that used a CAU control group, surprisingly, only 2 (5%) [71,80] detailed the care procedures involved. After all, CAU has been at best summarized as a blend of psychosocial and pharmacotherapy treatments. Regardless of the heterogeneity, it was suggested that the differential effect among subtypes of CAU was not significant [33]; therefore, we interpreted the effects of CAU as a collective therapeutic strength of existing services, and encouragingly, CAU was able to produce strong improvement in symptoms of depression in the medium term (g=−0.956), which is comparable with the low end of the within-group effect sizes as estimated in the active intervention arms in DPI studies [26].

The information control was an interesting type of control to consider given that reading educational materials is sometimes regarded as a stand-alone passive intervention. Our results showed that the information control was quite a strong intervention that produced a moderate effect immediately after the intervention period and in the short term. This is in line with previous meta-analytic findings showing that psychoeducational interventions were able to reduce symptoms of depression [154]. The effect of information controls shrank after 6 months in our study, indicating that the effect of passively receiving mental health–related information may dissipate over time. Although psychoeducation generally forms the core of most large-scale campaigns, for example, public awareness campaigns that disseminate mental health information, campaign holders may consider hosting booster sessions or setting up reminder prompts from time to time to maintain the effect in the long run.

Other control types, including sham, monitoring, and active controls, were also found to have moderate effects in the short term, and the effect was larger than that of waitlist controls. We acknowledge that the number of studies in each control category was very small and may have been subject to a small study bias. Nonetheless, the pattern of results points to the possibility that, regardless of the content of the control condition that the participants received, engagement or taking some action may help alleviate depression under the principle of behavioral activation, and these approaches fared better than simply waiting.

Subgroup analysis results also showed that, regardless of baseline severity, depressive symptoms reduced over time, although participants who started off with mild levels of severity consistently recorded a smaller effect size, possibly because of the floor effect. Studies in which there were moderate levels of depressive symptoms at baseline were associated with a steeper drop in symptoms until the medium term, and symptoms then bounced back in the long term. This implies that the participants in those studies may have reached the maximum potential for spontaneous recovery at approximately 48 weeks. Finally, the strongest effect noted in the severe subgroup beyond 48 weeks could be attributed to the exceptional efficacy of the CAU control type that dominated in this set of studies being analyzed.

As with any study, this meta-analysis had several limitations. First, the use of pre-post (within-subject) SMD may result in biased outcomes [155]. A reason is that the estimation of pre-post SMD depends on the correlation between pre-post outcomes, which is often not reported in studies. In our meta-analysis, we used a conservative correlation of r=0.59, which was based on the median within-group correlation found across 123 studies in the study by Balk et al [42]. However, given that each study may vary in the correlation estimate, there may be measurement error. Second, the quality of the included studies was not optimal. Almost all the studies (91/107, 85%) demonstrated some concerns associated with blinding issues given the nature of psychological interventions, be they face to face or digital [156], and possible underreporting of blinding procedures [157]. Only 4.7% (5/107) could be judged as high-quality, which was disappointing but understandable. It has been suggested that studies of poor quality generally inflate their effect sizes [158,159]; therefore, our interpretation of the findings should be taken with caution. Although it is difficult to truly blind participants in DPIs, blinding other key persons, including data managers, statisticians, and conclusion makers, is comparatively practical. Nonetheless, researchers should make an effort to blind all parties whenever possible. Mataix-Cols and Andersson [160] recently detailed 10 practical recommendations on blinding that future DPI studies should attend to.

In addition, the high degree of heterogeneity (>75%) among almost all sets of analyses made interpretation very difficult [44]. Despite our efforts to regress the overall effects on age, gender, and education, these individual characteristics were insufficient to explain the heterogeneity. A possible reason for this high heterogeneity may be the use of a relatively broad inclusion criterion that encompassed people with any depressive symptoms along the spectrum, even though studies targeting obvious comorbid conditions, for instance, physical medical conditions, posttraumatic stress disorders, and psychotic disorders, had already been excluded. Nonetheless, we included studies with comorbid anxiety given that it is highly common and can enhance the external validity of our findings. Future analyses may try to narrow down the depression subtypes and focus on people without comorbid conditions.

Some may believe that there is no point in pooling effect sizes of such different conditions and that effect sizes should not be pooled in case of substantial statistical heterogeneity. However, we argue that those who agree to participate in a DPI RCT represent an overarching group of individuals that share a similar mentality, for instance, being open to receiving interventions and self-motivated to change. Moreover, the high heterogeneity provided important insights on the documentation of control procedures and contents. We also recommend future trials to gather information regarding external help-seeking behaviors exhibited by control participants during the study period.

Despite these limitations, this study is the first comprehensive meta-analytic review that focuses primarily on the control groups of DPI studies. It provides valuable information regarding the magnitude of improvements in depressive symptoms in various control conditions over time. We quantified the within–control-group effect sizes among a fairly large number of DPI RCTs. Our findings have important implications for intervention research and practice. In psychosocial intervention research, studies that adopt a quasi-experimental design might overstate the treatment efficacy by not being able to account for the highly significant nontreatment effects, as quantified in this study. To uphold scientific rigor, we advocate that an RCT design should be perceived as the minimum requirement, not a high standard, for conducting intervention research. Strictly speaking, a quasi-experimental design should be abandoned for this type of research.

This study also provided evidence that depressive symptoms generally decrease over time. The severity of the symptoms reduced significantly within a few weeks even without structural treatment. Here, we must clarify that it is not our intention to discourage help-seeking behaviors or discredit the values of established evidence-based treatments. Instead, we hope that, by informing of a natural remitting trajectory of depression, researchers and practitioners and, most importantly, people who are experiencing various levels of depressive symptoms themselves do not see depression as a never-changing condition and negate the possibility of spontaneous recovery. They may consider adopting watchful waiting when active mental health services are not readily available given that waitlists were found to be efficacious in the short term.

In summary, this meta-analysis focused on the effects of control groups in DPI studies. The results concurred with previous findings on face-to-face psychotherapy studies that control conditions were able to produce small to moderate reductions in depressive symptoms within and beyond 48 weeks. Future studies should continue to investigate the nonspecific effects in intervention studies and explore meaningful moderators to explain the heterogeneity in DPI studies.