We conducted a qualitative content analysis of all intervention pages (ie, all app content) in Mindset, a therapist-guided smartphone CBT intervention for depression.

The original open trial of Mindset for Depression took place at Massachusetts General Hospital in Boston, Massachusetts, between May 2022 and February 2023. It was approved by the Institutional Review Board of Massachusetts General Hospital (protocol 2020P001958) and was registered on ClinicalTrials.gov (NCT05386329). All trial participants provided informed consent before the initiation of study procedures and were given the option to withdraw at any time. Participants in the original trial were compensated US $25 at each of 3 assessment time points (midtreatment, posttreatment, and 3-month follow-up). Given that it was a content analysis of the intervention materials only, this study did not include any participant data, user-generated content, or personally identifiable information. Therefore, no additional consent was collected. No images, screenshots, or other materials in this paper or supplementary files contain any participant data or identifiable information; all content reflects only the intervention’s standardized therapeutic materials. The full open trial, including a detailed description of study methods and results, is reported elsewhere [46].

Mindset is a therapist-guided smartphone CBT intervention for depression consisting of 8 modules delivered across 8 weeks [46]. The intervention is grounded in CBT theory, which posits that improvements in depression are facilitated by changes in thoughts, behaviors, and emotional responses [50].

Participants had immediate access to the intervention after completing baseline assessments. They were instructed to use the app independently while attending weekly 15‐ to 20-minute therapist sessions conducted via videoconferencing (Health Insurance Portability and Accountability Act [HIPAA]-compliant) to review progress and receive support. Therapists were licensed clinical psychologists trained in CBT for depression, and, because of licensure requirements, participants were required to reside in Massachusetts.

The program began with a brief onboarding sequence that introduced CBT principles and app navigation (Table 1). Modules followed a typical CBT progression, beginning with psychoeducation and self-monitoring, moving into behavioral activation, and ending with cognitive restructuring and relapse prevention [50]. Module 1 could be completed at the participant’s own pace, whereas Modules 2‐8 required a minimum of 7 days per module to support practice and skill consolidation. During Module 2, participants were prompted to self-monitor by logging at least 3 activities they engaged in and the corresponding moods each day. From Module 3 onward, participants implemented behavioral activation by scheduling at least 3 activities per week aligned with 7 value domains (health and wellness, causes and community, spirituality, creative pursuits, career and education, personal relationships, and day-to-day tasks), then logging mood after each activity. The activity library included 102 options, and activity scheduling continued through Modules 4‐8. Although therapeutic content was standardized, participants’ activity choices and the extent to which they revisited the activity scheduling and logging pages allowed for optional individual tailoring or personalization.

All data and therapeutic materials were sourced directly from the Mindset app, which included 393 unique app pages across onboarding content and modules (Table 1), totaling 943 pages when repeated content was included. No materials or pages from the app were excluded from analysis. Content was accessible only within the smartphone app and was not available as external files or in print. On average, there were 39.3 pages per module (SD 44.8). Content was delivered in the app via text, though some pages, such as the introductory module, mindfulness and grounding exercises, included the option to watch the content via video.

We used a dual-coding approach to systematically identify what therapeutic strategies were delivered and how. This approach, which we call mechanism mapping, pairs established CBT technique definitions [31,50] with the BCTTv1 [34] to describe the therapeutic content included and its behavioral operationalization.

We first identified and labeled CBT techniques present in Mindset to determine which therapeutic strategies were included and to evaluate alignment with established CBT protocols for depression (Table S1 in Multimedia Appendix 1). The CBT coding framework consisted of 16 techniques found in manualized CBT protocols for depression [25,31,50]. Refer to Table S1 in Multimedia Appendix 1 for coding definitions.

We applied the BCTTv1, a validated framework comprising 93 techniques organized into 16 hierarchically clustered categories [34,51,52]. The first author (GKJ) completed training through the official online platform [51]. The 16 categories and their definitions are fully described by Michie et al [34] and in Table S2 in Multimedia Appendix 1.

We chose BCTTv1 because of its ability to characterize the “ingredients” that operationalize change in a digital CBT intervention, consistent with a mechanisms-focused approach [34]. It also provides a validated, fine-grained taxonomy for identifying discrete, observable techniques embedded in app-based content, that is, what users see and do on each screen, making it particularly appropriate for therapy apps that translate evidence-based strategies into modular, self-guided activities. Other frameworks, such as the Theoretical Domains Framework and the Capability, Opportunity, Motivation–Behavior (COM-B) theory, describe why behavior changes by organizing determinants, but do not enumerate concrete techniques [53,54]. Although BCTTv1 was originally developed for health behavior interventions, its structure aligns well with CBT’s emphasis on observable behavioral and cognitive skills [50].

The raw text of the Mindset app was organized page-by-page to preserve the intervention’s structure during coding. A collaborative coding approach was used to identify CBT techniques and BCTs [55]. Techniques were coded based on the presence of explicit content, exercises, or instructions; implied techniques were not coded.

Coding was conducted by 2 doctoral-level clinical psychologists (GKJ and JYS) and 1 bachelor’s-level research assistant (HTA), all familiar with CBT for depression. None were involved in the initial development or testing of the Mindset intervention. The first author (GKJ) independently coded each intervention page in Dedoose (SocioCultural Research Consultants, LLC) [56]. Two additional coders (JYS and HTA) reviewed the codes and documented disagreements in memos. Discrepancies were discussed collaboratively using the memos as well as CBT and BCTTv1 definitions until consensus was reached.

To assess coder agreement, we report percentage agreement as the reliability metric [57]. Percentage agreement is appropriate for qualitative content analysis with multiple potential codes (eg, different pages of an intervention included multiple techniques or skills), because kappa statistics can be overly conservative when the number of possible categories is large relative to the sample size [57,58]. Before consensus, agreement was 93.62% for BCT codes (822/878) and 93.75% for CBT codes (494/528).

We calculated the frequency and distribution of CBT techniques and BCT categories across Mindset, including mean applications, SD, range, and percentage of total app pages.

For ease of interpreting results, we summarize BCT categories rather than each technique, and we prioritize CBT techniques and BCT categories that contribute ≥10% of total codes for reporting.

To prevent repeated content from inflating counts, we weighted pages that participants were guided to revisit multiple times. For example, activity scheduling pages appeared in Modules 3‐8, with instructions to schedule at least 3 activities per week. We applied a weighting factor equal to the average number of activities scheduled per participant divided by the total number of possible activity pages (n=110). This weighting assumption was based on participant instructions provided in the app. After applying this weighting factor, the 110 activity library pages in each of Modules 3-8 were reduced to 7 weighted pages per module, yielding an analytic dataset of 325 pages (Table 1) used as the denominator for all frequency and percentage calculations.

Data aggregation and analysis were conducted using R version 4.4.2 (R Core Team) [59]; a heatmap was developed using the pheatmap package [60], and figures were finalized in Canva (Canva Pty Ltd) [61]. Multimedia Appendix 1 includes detailed documentation of coding distributions.

This study follows the TIDieR (Template for Intervention Description and Replication) reporting guideline to promote transparency and replicability in describing intervention content and delivery [62]. A completed TIDieR checklist is provided in Checklist 1.

All 16 CBT techniques were represented and applied 528 times (mean 66.0 per module, SD 56.0) throughout the intervention. Figure 1 shows the distribution of CBT techniques by module, and Table 2 reports the descriptive summary of identified strategies. Full frequencies and distributions are reported in Multimedia Appendix 1.

Psychoeducation was present in all modules and accounted for more than half of the intervention pages (164/325, 50.5%), followed by skill building (110/325, 33.8%) and cognitive restructuring (46/325, 14.2%), both of which were most present in Module 1. Activity scheduling (42/325, 12.9%) appeared consistently across Modules 3‐8, in alignment with repeated practice of behavioral activation skills. Self-monitoring (39/325, 12.0%) was present intermittently throughout the intervention, but most focused on modules that asked users to log and observe mood changes linked to activities.

Homework assignments, self-kindness, problem-solving, and value identification appeared on 3%‐10% of the pages. Labeling emotions, mindfulness, guided imagery, and relapse prevention were less frequent (<5%).

The intervention included 37 BCTs (39.8% of the taxonomy), from 13 of 16 categories, applied 878 times (mean 109.8, SD 92.0) across modules. Figure 2 shows the distribution of BCTs by module, and Table 3 provides a descriptive summary of the identified techniques.

Six BCT categories were found in all 8 modules: shaping knowledge (205/325, 63.1%), repetition and substitution (138/325, 42.5%), feedback and monitoring (113/325, 34.8%), self-belief (89/325, 27.4%), comparison of behavior (58/325, 17.8%), and associations (51/878, 15.7%). Goals and planning (87/325 26.8%) and identity (58/325, 17.8%) were present in all modules, except Module 2, where participants began learning to monitor thoughts and activities. Natural consequences (51/325, 15.7%) were present in more than half of the modules.

Comparison of outcomes, social support, antecedents, and reward and threat occurred on fewer than 5% of pages. No techniques were coded from the regulation, scheduled-consequences, or covert-learning categories.

We examined how CBT techniques were behaviorally implemented through the codelivery of BCT mechanisms at the page level. Each co-occurrence corresponds to an intervention page where a CBT technique and a BCT are presented simultaneously, aiming to provide insight into how skills are taught, rehearsed, and reinforced during intervention use (Figure 3).

Psychoeducation showed the largest overlap, co-occurring with all 13 observed BCT categories, most frequently with shaping knowledge (119/325, 36.6% of intervention pages had this overlap). Skill building was also widely scaffolded, most often through repetition and substitution (114/325, 35.1%), feedback and monitoring (83/325, 25.5%), and shaping knowledge (80/325, 24.6%). Activity scheduling was consistently behaviorally reinforced, most prominently through goals and planning (72/325, 22.2%), repetition and substitution (42/325, 12.9%), and shaping knowledge (42/325, 12.9%).

Several core CBT techniques were behaviorally supported at moderate densities. For example, cognitive restructuring was frequently paired with repetition and substitution (43/325, 13.2%) and feedback and monitoring (35/325, 10.8%). Self-monitoring and problem solving were supported by fewer BCT categories (generally 2%‐8% of pages per pairing).

By contrast, mindfulness, guided imagery, and relapse prevention were rarely paired with BCTs, with most co-occurrences appearing on ≤10 pages (≤3.1%). Motivational, consequence-based, and social mechanisms, including reward and threat, social support, comparison of outcomes, and antecedents, were similarly infrequent (≤20 pages, ≤6.2% each).

This study aimed to advance transparency and mechanistic understanding in digital mental health by conducting a comprehensive analysis of Mindset, a therapist-guided smartphone CBT intervention for depression. Three primary findings emerged. First, all 16 CBT techniques were present across the intervention’s 8 modules, with a total of 528 instances. Second, Mindset incorporated 37 unique BCTs from 13 of 16 taxonomy categories, coded 878 times throughout the intervention. Shaping knowledge, repetition and substitution, and feedback and monitoring were most prevalent. Third, co-occurrence analysis highlighted the behavioral mechanisms through which CBT strategies were operationalized. High-frequency techniques such as psychoeducation, skill building, and activity scheduling were consistently paired with multiple BCTs that structure practice (repetition and substitution), provide feedback (feedback and monitoring), and support learning (shaping knowledge). In contrast, lower-frequency techniques such as mindfulness and guided imagery showed minimal BCT scaffolding, suggesting they were introduced but not behaviorally reinforced through repeated practice. These findings show how Mindset accomplishes fidelity to CBT protocols through multilayered behavioral mechanisms while establishing a replicable methodology for describing therapeutic content in a digital intervention.

This study introduces a mechanism mapping approach that directly addresses limitations of prior evaluations of digital CBT by methodologically examining how therapeutic techniques are implemented in user-facing content [25,43,44,63,64]. Specifically, our co-occurrence analysis (Figure 3) builds on previous studies by showing which BCTs scaffold each CBT strategy, making implementation depth visible and quantifiable.

Applying this mechanism mapping approach to Mindset highlighted considerable variation in implementation depth across CBT strategies. High-frequency techniques showed dense scaffolding: psychoeducation drew upon all 13 BCT categories, engaging users through informational content (shaping knowledge), emotional reflection exercises (natural consequences), and repeated practice activities (repetition and substitution). In contrast, mindfulness and guided imagery appeared with minimal BCT support, suggesting in-app introduction without behavioral reinforcement. The content distribution also showed app design choices: Module 1 was the largest (135 pages vs 9‐28 in other modules) and most densely scaffolded, front-loading introductory skills with instruction, practice, and feedback, laying the groundwork for subsequent modules to build upon.

This variation in implementation would be invisible using CBT-only classification but may help explain variability among apps claiming to deliver “CBT-based” care. Evidence from component-focused meta-analyses reinforces why implementation depth matters: combined cognitive and behavioral processes mediate digital intervention outcomes, but techniques delivered on their own showed no effects [45]. Additionally, skill enactment, actively practicing therapeutic strategies, predicts clinical improvement in most trials, whereas knowledge acquisition alone does not [32]. In Mindset, the BCTs supporting behavioral activation and cognitive change, such as instruction, repeated practice, and ongoing monitoring, appeared consistently across modules, creating structured opportunities for skill rehearsal. The intervention was designed this way so that users actively had to practice and rehearse strategies, not just learn about them. Most importantly, mechanism mapping generates testable hypotheses about content features that may facilitate clinical effects. To our knowledge, no other digital mental health study has visualized or described these relationships, providing a novel analytical framework that is replicable across interventions.

Mindset’s 16 CBT strategies and 37 BCTs exceed typical implementation in digital mental health tools. Prior assessments show that most digital tools include only a median of 3‐4 CBT-based elements [25,29] and 9‐15 BCTs, with conversational agents averaging 15 BCTs (range 4‐30) [63], eating disorder interventions averaging 14 BCTs (range 9‐18) [43], and general mental health apps containing 1‐10 techniques [65]. Our higher technique count compared to prior reviews likely reflects methodological differences in content analysis. We examined every page of the actual intervention, where previous studies coded app store descriptions or published summaries [41,44,63,64], which risks overlooking embedded therapeutic elements and overrelying on developer claims. However, technique quantity alone is not a reliable predictor of treatment effects [65]. Interventions grounded in established psychological theory show better outcomes than those lacking theoretical grounding [43], and full CBT packages outperform single-component strategies. Accordingly, Mindset’s extensive technique involvement may matter less than how these techniques are combined and delivered.

The most frequent CBT techniques in Mindset align with what systematic reviews consistently identify in digital interventions: psychoeducation (present in 50%‐75% of apps), behavioral activation including activity planning and scheduling (31%‐68%), and cognitive restructuring (31%‐79%) [25,27,29]. However, many techniques we identified in Mindset are inconsistently reported in large-scale content syntheses. Skill building, for example, appears in 0%‐12% of interventions depending on how broadly it is defined, with some reviews only capturing specific skill types such as social skills training [25,27]. Similarly, techniques such as homework assignment, self-compassion or expressing self-kindness, problem-solving, stimulus control, and emotion labeling were present in Mindset but are rarely systematically documented in existing app reviews. This inconsistency may reflect the absence of standardized CBT technique taxonomies in digital mental health research, making cross-study comparisons difficult and potentially underestimating the therapeutic content actually delivered to users.

The BCTs most strongly associated with engagement in mobile health interventions, feedback on behavior, self-monitoring, and instructions on how to perform a behavior were among Mindset’s most frequently used [41]. Importantly, these behavioral strategies were not features exclusively designed to keep users engaged; they were directly tied to CBT skill enactment (eg, mood monitoring implemented in service of behavioral activation). This integration contrasts sharply with common commercial designs in which self-monitoring or reminders are detached from core therapeutic mechanisms [25,41]. Mindset’s approach of embedding engagement-related BCTs within therapeutic workflows may explain how the intervention maintains both user engagement and therapeutic fidelity.

Our comprehensive content analysis also sheds light on which CBT strategies received minimal behavioral support and which BCT categories were absent entirely. Techniques such as grounding and mindfulness appeared infrequently and were associated with few BCTs. However, this may reflect the implementation format rather than inadequate support: these strategies were delivered through video demonstrations, which our text-based coding may have undercounted compared to written exercises that explicitly name behavioral techniques. Additionally, entire BCT categories were absent from Mindset, including regulation (eg, reduce negative emotions and conserve mental resources) and scheduled consequences (eg, punishment and reward). However, transparency about what is missing must be interpreted cautiously. Identifying absent strategies or mechanisms does not indicate whether their inclusion would improve outcomes. The value of this analysis lies in enabling transparent evaluation of what tools actually deliver, allowing clinicians and researchers to generate testable hypotheses about which gaps, if any, meaningfully impact outcomes.

This analysis has several limitations. First, the study did not statistically link participants’ exposure to or adherence with these techniques to clinical outcomes, preventing examination of which techniques may have been associated with these outcomes. Second, the review focused on text-based content, potentially underestimating exposure to interactive audio or video elements that may be more engaging than their script counterparts. Third, despite formal BCT training, content coding may have introduced subjective biases. Fourth, Mindset includes therapist-delivered content that was intentionally excluded to isolate digital components, meaning this analysis may underrepresent the full therapeutic exposure users received. Finally, our analysis concentrated on evidence-based CBT techniques and BCTs, without assessing non–evidence-based or potentially contraindicated content. Some mental health apps include elements that may conflict with recommended approaches, which may also impact safety and effectiveness. Accordingly, mechanism mapping should therefore be seen as an initial foundation for objective fidelity monitoring instead of a complete measure of treatment quality.

This study demonstrates that transparent evaluation of digital mental health interventions is feasible and necessary. We offer the following actionable recommendations for advancing quality and accountability in the field. For intervention developers, we recommend auditing existing tools using this dual-coding framework to identify which therapeutic techniques lack behavioral scaffolding so that we can confirm techniques are appropriately reinforced through user action. For researchers, we recommend contributing to transparency by reporting the frequency of therapeutic techniques, making this standard practice to enable meta-analyses to examine implementation depth as a moderator of outcomes. Our methodology is fully replicable for any text-based intervention. We encourage clinicians to evaluate apps based on documented content rather than marketing claims, to seek evidence of technique coverage and reinforcement of mechanisms, and transparent reporting of delivered content. This can be used to inform patient-centered recommendations of these tools. Finally, for the field, we hope that this report sets a new precedent for minimum reporting standards when evaluating these tools, so that they can be more comprehensively evaluated and compared.

The rapid global adoption of digital mental health tools has outpaced our ability to evaluate what they actually deliver. This study introduces mechanism mapping as the first systematic approach to describe which therapeutic strategies are present and how they are behaviorally operationalized through user actions. Unlike prior content analyses that rely on presence or absence coding, our dual-framework methodology reveals implementation depth, distinguishing between techniques that are mentioned versus those supported through repeated practice, feedback, and skill consolidation. This innovation addresses a fundamental transparency gap: most digital interventions function as “black boxes,” making it impossible for clinicians, patients, or researchers to assess therapeutic fidelity.

This replicable framework brings 3 critical contributions to the field: it enables comparative evaluation of digital interventions based on therapeutic content rather than marketing claims, provides testable hypotheses about which implementation features drive clinical effects, and establishes a methodological standard for transparent reporting. In the real world, this work supports clinicians in making evidence-based referrals, empowers patients to choose interventions aligned with their therapeutic needs, and guides developers in creating tools with demonstrable fidelity to evidence-based practices.

Future work should extend this approach by integrating behavioral measurement and mechanistic outcomes, applying mechanism mapping across multiple digital interventions to identify common implementation signatures, exploring whether implementation patterns could inform adaptive tailoring of behavioral supports, and incorporating therapist-delivered components in hybrid models to capture the full therapeutic ecosystem. Importantly, transparency must become the standard in digital mental health, not the exception. Understanding how apps work, for whom, and through what processes is critical for the next generation of precision mental health care.