Our review used realist synthesis philosophy and principles as recommended by Pawson and Tilley [14] and is reported using Realist And Meta-narrative Evidence Synthesis: Evolving Standards (RAMESES) criteria [15]. Realist synthesis is an approach concerned with theory development and refinement of “how interventions work, for whom, and under what circumstances” [16]. From a realist lens, treatment effects are understood to be influenced by implementation context, and thus, Context-Mechanism-Outcome (C-M-O) configurations provide potential explanations about causal processes for treatment outcomes. Realist synthesis is also particularly useful when reviewing a complex and heterogeneous body of research (ie, diverse study designs, interventions used, outcomes measured) [14]. In this realist review, we examined the relationships between contextual factors for Internet-based CBT and IPT delivery (eg, conditions of use and the type of user, therapeutic content) and adolescent outcomes (eg, symptom reduction, satisfaction, therapy adherence), and the underlying mechanisms (eg, user behaviors, technological features/system design) that connect them.

Many realist reviews begin by identifying a theory or theories to develop a preliminary list of C-M-O configurations. The evidence identified in the review is then used to determine which C-M-O configurations are upheld when reviewing the evidence. In contrast, we identified potential theories for our review using an iterative process during project development—brainstorming within the review team and reviewing literature on human-technology interaction and studies of Internet-based psychological therapies for adolescent depression. Persuasive System Design (PSD) emerged from this process as a key framework for our C-M-O configurations. Being derived from both behavior change models and information technology systems models, PSD provided a comprehensive framework for exploring how systems influence the attitudes and behaviors of system users [17,18]. The framework is composed of four system features that relate to the persuasiveness of an Internet-based therapy—primary task support, dialogue support, system credibility support, and social support (Table 1). An Internet-based therapy can take on a persuasive role and use persuasive mechanisms in online delivery (persuasive system design) by (1) conveying symbolic (eg, text, data graphs, icons) and sensory content (eg, real-time video, virtual worlds, simulation) and (2) facilitating a social experience through the adoption of animate characteristics (ie, physical features, emotions, voice communication) and roles (eg, coach, pet, assistant, opponent), and using social dynamics (eg, greetings, apologies, taking turns) [17].

Using the PSD model, we then theorized what persuasive system features were likely to be linked to each outcome (ie, which features were mechanisms) reported for Internet-based CBT and IPT, and what delivery contexts were the most relevant in allowing that to happen. The end result of this discussion was a list of C-M-O configurations that would guide data analysis.

Search for Relevant Literature Realist methods call for a purposive and iterative approach. The search process began with a research librarian conducting a systematic search of academic databases in the psychology and health fields: Medline, CINAHL, Embase, PsycINFO for the period 2000-2016. Other search engines—Google and Beacon—and gray literature repositories (ACM Digital Library, OpenGrey, Canadian Agency for Drugs and Technologies in Health, IEEE Digital Library) were manually searched to identify gray literature such as government reports, community program evaluations, and conference proceedings for the same time period. Search terms were related to clinical area (ie, depression, mental health), modality (ie, online, Internet-based), and therapeutic approach (ie, CBT and IPT). The strategies for two searches are provided in Multimedia Appendix 1.

We also manually searched the table of contents in medical informatics journals (ie, Journal of Medical Internet Research, Internet Interventions, Journal of Cybertherapy and Rehabilitation, Journal of Telemedicine and Telecare). Snowball searching was also conducted (ie, reviewing eligible article reference lists) to identify relevant documents that may have been missed in the search process.

Search results from academic databases were downloaded into Endnote (Thomson Reuters, Version 7.2) and then screened for eligibility by 2 trained raters (authors LW, AR). Inclusion criteria were as follows: (1) intervention studies (eg, clinical trials) were eligible for inclusion if they evaluated Internet-based CBT or IPT with adolescents with depression, (2) theoretical papers, mixed-methods and qualitative studies, and policy/implementation documents were eligible if they included a focus on how Internet-based CBT/IPT is proposed to work for adolescents with depression/depressive symptoms, and (3) documents that met Criteria 1 and 2 were eligible if they were published in English language. Any reviews (systematic, meta-analysis, etc) identified during our screening process were appraised to identify intervention studies and other potentially relevant documents.

During Stage 1 screening, the eligibility of a random subset (10 citations) was assessed independently by 2 team members (authors LW, AR), and interrater agreement was assessed within the “substantial” Kappa range (Cohen kappa=.74).

The evidence for each Internet-based therapy was assessed for relevance and rigor by consensus of 2 reviewers (authors LW, AR). These two reviewers also conducted co-coding and debriefing activities periodically during analysis.

Relevance was defined as the level of contribution to the review, and rigor was defined by the methodological quality of a study conducted on the Internet-based therapy (intervention, mixed-methods, and qualitative studies). Relevance was assessed by reviewing the details provided for an Internet-based therapy’s (1) context (eg, user, program features/design components), (2) mechanism(s): hypotheses as to how specific elements of the therapy worked, was proposed to work, or did not work, and (3) outcomes: reasons for therapy effect or lack of effect on specific adolescent outcomes. These details were obtained by reviewing documentation of usability evaluation, therapy/study protocols, and publications related to evaluations (eg, clinical intervention studies evaluating efficacy or effectiveness). Relevance was rated as low/none (no or little information), medium (some information), and high (well-described information).

The methodological quality of evidence (rigor) around each therapy was assessed, where possible, using the Mixed Methods Appraisal Tool (MMAT), an effective and practical quality assessment tool [19]. The tool includes sections for qualitative, mixed-methods, and quantitative studies. MMAT scores range from 0 (no criteria met) to 4 (all criteria met). Pawson describes the process of determining rigor as whether a “particular inference drawn by the authors has sufficient evidence to make a methodologically credible contribution to the test of a theory” [20]. Thus, we used the MMAT to assess the credibility of the reported findings based on the methodology described. Following MMAT guidelines, if publications were companion papers on the same data set, they were assessed as a set of publications.

NVivo software (QSR International; Version 11) was used to extract data. We extracted data for several aspects of the context of the Internet-based therapy: (1) user context (eg, urban/rural, age group, sociocultural composition, clinical severity), (2) usage context (eg, therapy objectives, adjunct versus stand-alone therapy), and (3) technology context (eg, synchronicity, use of multimedia, software and bandwidth requirements, mobile phone versus desktop). We also extracted information on therapy design (ie, conditions under which an adolescent completed the therapy: sequence, structure, timing; CBT/IPT features). In instances where the description of the therapeutic content was not available in the article, we included information from the article’s citations that described a therapy. We also extracted available information on PSD system features (primary task support, dialogue support, system credibility support, social support), therapy usage (eg, attrition, engagement), as well as clinical (eg, symptom reduction) and therapeutic (eg, therapeutic alliance) outcomes. Where available, information on full or partial C-M-O configurations was also extracted for individual therapies (ie, we sought data on particular therapies that could explain what Mechanism led to an Outcome, under which Contexts).

To promote consistency during data extraction, a coding guide with operational definitions for each code was used. PSD principles were used to guide the coding of mechanisms [14,18]. Two review team members (authors LW, AR) cross-referenced decisions regarding coding and extraction on a random subset of 10 articles and the remaining documents were coded by 1 reviewer (author LW).

We used a multistep approach to identify and organize information about what contexts of Internet-based CBT/IPT and persuasive system design attributes may contribute to adolescent outcomes. Our initial list of C-M-O configurations was revised based on consensus between team members. Drawing from qualitative synthesis methods, we selected an “index” therapy (CATCH-IT [21]) that was conceptually rich (ie, high methodological rigor, and most complete and robust descriptions) and could be used as a starting point for lines of argument synthesis [22] (ie, a form of grounded theorizing). Using a cross-case comparison method, we initially compared each therapy to the index therapy using the data matrix produced during data extraction. Similar and recurring concepts and themes were pulled together in order to corroborate or refute the proposed C-M-O configurations. This process involved identifying concepts from one therapy and recognizing the same concepts in another therapy, though they may not be expressed using identical words [23]. Proposed C-M-O configurations were analyzed at different levels of abstraction (ie, within and across therapies) to determine the most robust and plausible explanations of how in a context, with a mechanism, outcomes could be generated.

As a final step we compiled a framework matrix for each C-M-O configuration in NVivo in order to map “demi-regularities”, semi-predictable patterns of therapy outcomes [16]. This allowed the review team to examine which therapies provided what evidence to support each C-M-O configuration. Conference calls with team members were used to discuss, amend, or confirm identified patterns. In instances where a C-M-O configuration was judged to not fully explain the demi-regularities, we returned to the data to refine the configuration.

Details of the level of contribution and quality assessments are provided in Table 3. Based on the level of contribution assessment, three therapies—MoodGym, SPARX, and CATCH-IT—were rated as having high contributions to all three dimensions relevant to the review questions: contexts, mechanisms, and outcomes. Twelve other therapies provided medium to high contributions to at least one of the three dimensions. Contributions from four therapies were low. Across the therapies, descriptions of mechanisms (how technology components were expected to function in relation to outcomes) and context were less developed than descriptions of outcomes. The methodological quality of studies associated with each therapy varied considerably. Half (28/57) of the documents were research studies eligible for MMAT ratings—15 randomized controlled trials, 3 nonrandomized trials, 3 quantitative descriptive studies, 4 mixed-method studies, and 3 qualitative studies. All studies met at least 2 of four MMAT criteria. Studies with lower ratings did not provide a clear description of the randomization process, had higher than 20% study dropout rates, or used nonvalidated measurement tools. Follow-up beyond 6 months was conducted only for the CATCH-IT program suggesting limited knowledge about post-program experiences of adolescents.

Of the candidate C-M-O configurations initially put forward using the PSD model, five configurations were substantively supported by available evidence (Table 4). We present the configurations with key examples of the contexts, mechanisms, and outcomes from the documents reviewed.

Advances in technology have allowed for health care programs to connect users to treatments in dynamic ways. The dramatic growth of technologies designed to persuade and motivate represents a significant shift in focus toward end-user computing in health behavior change therapies [18]. Internet-based therapies for adolescent depression are one group of therapies that have incorporated persuasive design strategies to influence therapy outcomes. However, to date, such programs have not been reviewed and evaluated with this perspective. Not accounting for human-technology interaction in therapy design and evaluation limits our understanding of how Internet-based therapies work. In our view, this review of Internet-based CBT and IPT for adolescent MDD highlights the need to purposefully consider PSD features early in the program design process and to further develop PSD theory to help explain therapy outcomes. This review is also an important step in understanding how PSD features of Internet-based therapies may work based on existing literature. The C-M-O configurations generated by this review represent important hypotheses. The testing of specific program mechanisms (ie, PSD features, provider involvement) through hypothesis-driven, dismantling approaches is now necessary to advance the understanding of the effect of Internet-based therapies on users.

A key argument in favor of developing and offering Internet-based psychological therapies to young people is increased access [57] to effective mental health care. Therapies in our review were either stand-alone therapies meant to increase options for young people with depression [69] or used in combination with existing health care [25] or school-based [58] services. Therapies that relied on existing services reported better engagement with the program and completion rates, although across all studies, attrition was a reported issue. Does availability matter if young people do not complete the program as intended? At this point in time, this question is hard to answer. It is necessary for future studies to examine whether young people stop using stand-alone therapies because of symptom improvement, changes in motivation, or because of the program features. Qualitative exploration of the type of support provided and required during adolescent use of programs used in combination with existing services would also provide valuable information on necessary “program ingredients.” While adherence to Internet-based programs has traditionally been low [85], the use of PSD principles, or in-person support from existing services at critical program timepoints (eg, when dropout has been shown to occur, when therapeutic content is strenuous for the user) may address program engagement and adherence.

The findings from our review, in terms of surface credibility and liking, suggest that adolescents’ visual experiences lead to esthetic and credibility judgments [86-88] and should also be taken into account. Bennett and Glasgow have argued that the most critical design gap in the current generation of Internet-based programs is the underuse of Web 2.0 features (eg, social media, user-generated content, collaborative consumption) [89]. In our review, many therapies were adapted from face-to-face or manualized programs. This may have resulted in a “replication” of components that were originally designed for face-to-face interactions rather than fully optimizing Web 2.0 features available for delivery.

Fogg has argued that persuasive systems will work only if the user has sufficient motivation and the user’s ability is being adequately triggered to perform the new behavior [17]. High rates of noncompletion of Internet-based psychological therapies for adolescent MDD suggest the need for therapies to address motivation in the context of depression/depressive symptoms (eg, if a depressed adolescent is not motivated to log into a stand-alone program, what program supports are needed for behavioral activation?). That the use of specific PSD features—reduction, tunneling, self-monitoring—led to increased adolescent engagement and satisfaction, and improvement in depressive symptoms and functioning, suggests the cognitive load of therapies may also need to be addressed. The advantage of primary task supports—tunneling, reduction, and self-monitoring—is that these PSD features can increase positive affect (ie, “I can do this”) [90,91]. Pairing primary task supports with other persuasive strategies such as self-monitoring, reminders, simulation-rehearsal, and suggestion-reward can be especially effective together [92,93]. Of the studies in this review, despite high attrition, most reported a positive clinical outcome associated with Internet-based program use, suggesting that we do not yet have a clear understanding of therapy “dosage” for adolescents and how “persuasive” a therapy needs to be (eg, the persuasive experience) to ensure optimal outcomes for adolescents. Future research is needed to strengthen our understanding of the relationship between treatment adherence and symptom change. Persuasive system design may be one testable model to facilitate knowledge generation. The studies in this review highlight that the relationship between engagement in therapy activities, adherence through to treatment completion, and treatment satisfaction are complexly related to clinical outcomes and require theoretical and empirical investigation.

Future studies should operationalize each PSD feature, hypothesize its intended effect, and measure its use and effect. As technology and methods of human-technology interaction evolve, this documentation and evidence will provide a valuable roadmap for the depression treatment field. Studies that use a factorial design or fractional factorial design would move the field forward by providing an opportunity to compare intervention groups that include multiple, and different combinations of, persuasive design functionality. For example, there could be conditions within both study arms in which some groups receive tailored feedback and others do not, some include a social networking forum and others do not, and some provide reminders and others do not. In this way, the impact of PSD features can be isolated. It might also be useful to develop hybrid designs that include both standard “randomization” as well as “preference” arms, in order to determine which groups of adolescents might be more attracted to certain PSD features.

The exploration of target population characteristics is also needed to determine how motivational (eg, readiness for change, self-regulatory skills), developmental/age-related, sociocultural, technical competency and modality features (synchronous, ambient, etc), and depression severity differentially interact with program mechanisms and impact adolescent outcomes. For example, research has shown gender differences in the perceived persuasiveness of numerous health intervention components, with females being more receptive to most persuasive behavior change strategies [94]. Under this approach, studies can identify mediators of therapy effect and examine both mediators and moderators as a means to identify program mechanisms (how) and for whom the therapy works best (who), respectively.

This review is the first to use a realist framework for studying Internet-based psychological therapies for adolescent depression. This framework allowed us to consider studies and theories together to understand how the therapies worked. We included numerous therapies identified in the gray literature, which allowed for a comprehensive appraisal of the current evidence base and reduced the risk of publication and selection bias. Previous reviews of Internet-based therapies for this population have focused only on empirical literature, and therefore, have provided limited insight into the complex causal pathways that may underpin therapy effects. Including multiple research designs, while challenging from a data integration standpoint, enabled the analysis to benefit from the strengths of each approach and corroborate findings across divergent contexts and theoretical orientations. From a realist perspective, this diversity has huge explanatory value and can help uncover contexts and conditions not typically captured in meta-analytic or traditional systematic reviews. In addition to offering a more thorough assessment of Internet-based therapies, this review supersedes existing reviews by including substantially more therapies and documenting the body of work around each one. A further strength of the review is the use of gold standard review methods (notably, duplication of screening, quality assessment, and consensus-building with research team members).

The greatest challenge in applying the PSD model is that no explicit heuristics have been defined for it yet, and so nuances between different PSD features are still being mapped [95]. The model would gain more strength from explicitly defined scales and instructions for evaluating the implementation of each PSD principle. A further challenge in exploring these therapies from a sociotechnological frame is that studies largely employed traditional clinical reporting methods with very little attention paid to describing the informatics architecture and the expected role of technology functionalities. In this regard, usability studies included in our review provided much needed information. Because we reviewed the literature and evidence base around these therapies, as opposed to conducting a workflow analysis of each therapy, our appraisal was limited to what authors reported, which may or may not be a full reflection of the therapy’s capabilities and design.

Another challenge in our review was the lack of information on the nature of adolescent’s interaction with PSD features. For example, current descriptions of dialogue support provided in Internet-based therapies are lacking. While authors described using email, reminders, and options for peer engagement, there was little detail about actual engagement with these features (eg, How often were reminder emails triggered?, How many adolescents elected to publish journal entries to their peers?, How many adolescents spontaneously emailed their assigned coach and how often?). In terms of peer-based dialogue support, some have argued that there is not yet very strong evidence for what type of peer-based social support therapies ought to provide [96].

Results from our review suggest there is room for improvement in both designing and implementing Internet-based therapies for adolescent depression and in elucidating how persuasive mechanisms are designed and ultimately function. We offer that many of the assumptions that implicitly shape Internet-based therapy development and delivery—adolescents are highly competent technology users, adolescents want to complete programs on their own, the more persuasive design components the better, or that compliance will result in improved outcomes—are vastly under-acknowledged and are based on pervasive assumptions about adolescents, what they prefer, and what they need. Improved engagement of adolescents with MDD in the design and development of future therapies is crucial if we hope to provide effective Internet-based therapies for this population.