Type 1 diabetes (T1D) often emerges during childhood or adolescence, affecting 1.5 million individuals aged <20 years, with an expected increasing prevalence over the next 20 years [1]. T1D requires lifelong, routine self-management activities and medical follow-ups to prevent acute and chronic diabetes complications and ensure stable health [2]. Emerging adulthood, a period of development spanning approximately from ages 18 to 28 years [3], is a particularly challenging period for those living with T1D not only because of a shifting dynamic in the responsibility for performing daily T1D self-management activities but also because it coincides with major life changes that introduce competing social, academic, and financial needs [4]. One of the common and observable risks introduced during this period is the transition from the pediatric to adult care system [5,6], which has been associated with care gaps that lead to adverse health outcomes and increased acute health care use [7]. There is a practical need for targeted interventions to support emerging adults living with T1D, equipping them with the knowledge and motivation to meet their T1D self-management needs alongside multifaceted and evolving psychological, educational, and vocational needs [7-9].

Increasing attention has been paid to the potential of digital health interventions in offering scalable support for people with T1D. Digital health interventions are widely accessible [10] and can incorporate multiple functions to meet the multifaceted needs of users [11]. Underpinned by a combination of different components, such as educational information [12-15], self-management tasks or appointment reminders [13,14,16], question and answer (Q&A) functions [17], and psychosocial support [17], some studies on digital health interventions in T1D have noted positive impacts on user acceptability [16], blood glucose monitoring behaviors [16], glycemic management [13,15], and clinic attendance [14,17]. Although promising, these results provide little insight into which of the multiple intervention components work to influence the desired outcomes, through which pathways, and under what circumstances [18]. The answer to this question is of relevance to health service researchers who are interested in intervention fidelity (eg, whether the intervention is implemented as intended) and policy makers who seek to understand how to maximize resource allocation amid growing financial constraints (eg, what elements are most effective and which groups are most likely to benefit) [19].

The UK Medical Research Council provides a framework for process evaluation that evaluates complex interventions by diving into the implementation process to understand intervention components, mechanisms of action, and the influencing context [20]. Realist evaluation, a specific type of process evaluation, is an explicit theory-driven evaluation framework that unveils the black box of the implementation process by specifying the context-intervention- mechanism-outcome (CIMO) configurations that could explain which components work (intervention and outcome), through which pathways (mechanisms), and for whom (contexts) [21,22]. In this study, we embedded a qualitative realist evaluation alongside a randomized controlled trial (RCT) that is investigating the effects of a tailored, multicomponent, text message–based digital health intervention (Keeping in Touch; KiT) on the transition experiences among emerging adults living with T1D. KiT integrates a broad range of validated intervention components, including educational content [4,12,13,15,23-26], transition coordination [17,27], appointment reminders [13,16,17], and psychosocial support [17], which have been delivered in both conventional structured programs and mobile-based interventions. By leveraging mobile technology, KiT overcomes the scalability challenges faced by conventional structured programs [28]. In addition, KiT stands out by offering more personalized content than many other mobile-based interventions [9], tailoring information based on users’ needs and interests, and providing customized answers through its Q&A feature. The complete realist evaluation protocol (including the hypothesized CIMO configurations) [29] and KiT intervention development process [30] have been published previously.

The objectives of this qualitative realist evaluation were to (1) qualitatively examine the hypothesized CIMO configurations outlined in the initial program theory to validate whether and how they may be activated by KiT and (2) refine the initial program theory based on insights from the intervention implementation process.

Combining the insights gained from the KiT program documentation and research articles on behavioral science theories for digital behavior change interventions, we previously identified 5 key KiT intervention components and 5 key intervention features, respectively [29]. These include the following:

By incorporating capability, opportunity, motivation, behavior model [31]; health action process approach [32]; and technology acceptance model [33] frameworks, we developed the initial program theory delineating the potential relationships among individual-level context, intervention components and features, mechanisms, and outcomes [29]. Therefore, we generated 10 hypotheses to understand which KiT intervention components and features may work, for which outcomes, and by which mechanism (Textbox 1).

The multicenter KiT trial is ongoing and is being conducted at 6 pediatric diabetes clinics in Canada—4 in Ontario and 2 in Quebec [30]. The multicenter trial was launched in January 2023 and finished participant recruitment in September 2024, with a 12-month intervention duration and an anticipated trial end date in September 2025. Details about the KiT trial design are available in the published protocol [30].

Ethics approval for this embedded realist evaluation was obtained from the Trillium Health Partners Research Ethics Board (ID 1086). Ethics for the broader KiT trial (trial registration number NCT05434754) was obtained from Clinical Trials Ontario through the Hospital for Sick Children Research Ethics Board (project ID 3986) and McGill University Health Centre Research Ethics Board (project number MP-37-2023-8823).

Participants provided written informed consent for the embedded realist evaluation during the trial enrollment. The research team contacted the KiT trial participants who provided consent with a letter of information and an invitation to participate in an interview. Participants received a CAD $25 (US $18.25) electronic gift card to complete the interview. Identifiable data were stored in password-protected files on secure servers and were only accessible by approved research personnel. Data deidentification was performed before analyses. This paper follows the reporting requirements outlined by the Realist and Meta-narrative Evidence Synthesis: Evolving Standards (RAMESES) II checklist (Multimedia Appendix 1) [34].

Participants were eligible for this secondary study (ie, embedded realist evaluation) if they met the following criteria: (1) were randomized to the intervention arm with completed baseline surveys, (2) had a minimum of 3 months of intervention use to allow them to receive the 2 optional educational topics (based on users’ needs and interests, respectively [29]), and (3) confirmed receipt of ≥2 text messages from KiT in the month before the interview to mitigate potential recall bias. These criteria allowed us to include participants who were sufficiently familiar with the intervention. We used purposive sampling to facilitate representativeness across participants’ sociodemographic characteristics and intervention use. We recruited eligible participants until we achieved sufficient information power, which aims at offering new contextualized insights that contribute to current knowledge [35]. This approach allowed us to answer our research questions and refine the initial program theory to inform future testing in the full trial cohort.

We performed a multistep qualitative data analysis following Gilmore et al [44], including interview data coding, case creation, and CIMO refinement.

This interdisciplinary implementation study reflected a collaborative effort among team members with diverse expertise and perspectives, including implementation science researchers; clinical scientists in endocrinology, pediatrics, and pediatric psychology; as well as an emerging adult patient partner with lived experience of T1D. We took part in this study based on a shared commitment to improving the accessibility and relevance of interventions that address the multifaceted and evolving needs of emerging adults living with T1D.

Throughout data analysis and interpretation, the research team considered their own positionalities and professional identities. The reflexive discussions and cross-checked interpretations reduced individual biases and enhanced the credibility and depth of our findings. This approach allowed us to generate more comprehensive, context-sensitive, and rigorous insights.

A total of 26 participants were invited to partake in the interview. Among them, 8 (31%) potential participants did not respond, and 2 (8%) agreed to participate but were lost to contact (ie, did not respond to follow-up messages to schedule the interview), leaving 16 (62%) participants recruited between July 2023 and May 2024. The interviews lasted 24 to 68 (mean 51, SD 10.4) minutes.

The 16 participants were aged 17 to 19 (median 17.9, IQR 17.8-18.1) years; 8 (50%) self-reported their sex as female and 8 (50%) as male. A total of 6 (38%) participants identified as White (with other major ethnic groups, including South Asian, Black, and Middle Eastern), and approximately two-thirds (n=10, 62%) of the participants had private insurance (Table 1; comparison with other intervention-arm participants who provided consent to participate in our study is available in Multimedia Appendix 3). Most participants responded to all 4 standardized questions and had a response rate of >80% to all questions sent by KiT from enrollment until their interview. Most participants received ≥4 educational topics, with driving, exercise, and nutrition and carbohydrates as the 3 most popular optional educational topics. Variability was observed in the use of optional KiT components, as expected, with a total of 12 (75%) participants using the appointment reminders, followed by the test reminders (n=7, 44%) as well as the note keeping and Q&A (n=2, 12%).

Qualitatively developing and synthesizing participant-specific CIMO configurations provided in-depth context into what KiT intervention features were being used, by whom, and via what mechanism. Taken together, these insights provided intervention-level insights into the different ways KiT may work, for whom, and how. Specifically, we found the following:

The importance of educational content (intervention) and reminders (intervention) as core intervention components aligns with previous research [51-53]. In contrast, the use of problem-solving support (intervention) may vary depending on participants’ awareness of available intervention components (context) and perceived knowledge or access to information (context). Similar challenges have been reported in previous studies [40,54], emphasizing the need for robust onboarding and routine reminders of available intervention components to improve engagement [40]. Similarly, the stress management strategies (intervention) are more likely to be effective for individuals with high baseline diabetes distress (context) and low confidence (self-efficacy [context]), reinforcing the value of personalized interventions that account for individual differences, such as duration of diabetes [55], baseline knowledge [56,57], and perceived needs [58].

These findings highlight the limitations of one-size-fits-all approaches, advocating for context-sensitive, multicomponent interventions that accommodate participant heterogeneity [57,59]. This requires identifying key subgroups, often described as archetypes [60], profiles [61], or personas [62]. Our results suggest that digital health interventions similar to KiT should consider tailored strategies for individuals with more recent diabetes diagnoses (duration of diabetes [context]), perceived limited access to credible knowledge (baseline knowledge or access to information, [context]), lower baseline self-efficacy (context), or higher baseline diabetes distress (owing to the recency of their diagnosis [context]). Tailored strategies could include educational content on T1D burnout and building coping strategies [63,64] and mood-enhancing messages (eg, self-affirmation, gratitude, and “mood booster”) [64]. In the meantime, further research is needed to better understand the specific needs of other subgroups of emerging adults living with T1D to inform targeted intervention design.

Previous studies have demonstrated that educational content (intervention) can serve as valuable reminders to nudge self-management behaviors [51,56,65], aligning with our results that reinforcement (mechanism) of existing knowledge might be as important as sharing new knowledge (mechanism) [59]. Desveaux et al [66] and Sandborg et al [67] illustrated that digital health interventions increased participants’ understanding of their disease condition (ie, knowledge [mechanism]), which translated to improved motivation and self-management (outcome). Even when some of the information is already known, reinforcing it (reinforcement [mechanism]) may still enhance self-management (outcome) [56]. Our work builds on and integrates these findings by specifying how knowledge (mechanism), reinforcement (mechanism), outcome expectancies (mechanism), self-efficacy (outcome), negative emotion (ie, diabetes distress [outcome]), and behavioral cueing (mechanism) may operate as chains of interconnected mechanisms. This interconnectedness of mechanisms reinforces the complexity of digital health interventions and advances our understanding of how their impact may be realized. By identifying knowledge (mechanism) and reinforcement (mechanism) as candidate upstream mechanisms, our results suggest that prioritizing these mechanisms is a high-yield consideration in designing digital health interventions to support chronic disease self-management (outcome). Taking it one step further, measuring knowledge (mechanism) as an upstream indicator of early impact may provide insight into whether a digital health intervention is well-positioned to achieve long-term goals (eg, glycemic management [outcome]).

Participants reported noticeable changes in mechanisms (knowledge) and proximal outcomes (self-efficacy and negative emotions due to diabetes distress) but reported limited perceived impact on the distal outcome—glycemic management (outcome). After ruling out technical problems with KiT’s intervention components design and delivery, we speculate that the lack of meaningful change in glycated hemoglobin (HbA_1c) may be an artifact of timing or reflect a mismatch between interventions and desired outcomes. First, text message–based interventions may require more time to influence distal outcomes such as HbA_1c [53], and participants in this study were interviewed during the first half of their intervention period. Previous studies have similarly noted meaningful changes in self-efficacy, negative emotions (ie, diabetes distress), and knowledge within 3 to 6 months of intervention exposure [53,68-70] and up to as early as 8 weeks [68], suggesting that this time frame is a reasonable window for proximal outcomes. However, while meta-analyses have shown digital health interventions can positively impact HbA_1c [71,72], the literature remains variable. For instance, Dobson et al [73] observed a significant reduction in HbA_1c at 9 months, while Franklin et al [13] and Castensøe-Seidenfaden et al [74] failed to observe an effect at 12 months. Our results suggest that the broader RCT (12 mo) may provide a clearer understanding of the minimum intervention length needed to see changes in HbA_1c. These findings reinforce the value of exploring upstream changes in the short term through realist evaluations.

Furthermore, while HbA_1c is widely adopted as a primary distal outcome in diabetes management interventions, previous literature has revealed its limitations in capturing short-term (<2 mo) changes in glycemic management and other important aspects such as quality of life [75]. These findings suggested the need to move beyond the reliance on HbA_1c alone. Furthermore, aligning with the broader KiT trial that incorporated additional outcomes, including transition readiness and self-efficacy, future research on similar interventions might benefit from incorporating additional outcomes, such as mean glucose levels [69], time-in range [75], and patient-reported quality of life [73], to provide a more comprehensive understanding of potential impact.

This embedded, theory-driven design contributes to both explanatory theory-building and pragmatic intervention strategies, illustrating how realist evaluation can facilitate the interpretability and utility of RCTs in implementation science.

First, while RCTs offer an unbiased estimate of intervention effectiveness, they typically provide only overall treatment effects without unpacking the implementation black box of how specific components of a multicomponent intervention work [14,17,76]. Embedded alongside the RCT, our realist evaluation filled this knowledge gap by uncovering the mechanisms through which particular intervention components function. This deepens understanding of the intervention’s inner workings and supports a more theoretically grounded interpretation of the trial results [77,78], particularly valuable for complex digital health interventions such as KiT [20].

Second, recognizing that many RCTs yield suboptimal results in distal outcomes [79,80], we timed our data collection alongside the trial. This approach enabled exploration of upstream mechanisms and proximal outcomes, revealing a cascading effect of KiT in which early changes were initiated, although distal clinical effects were not yet fully observable. By capturing early-stage signals, this approach offers a more comprehensive and nuanced understanding of how the intervention works over time. In the meantime, it also prompts reflection on whether we are measuring the right outcomes at the right time, which is an important consideration for fair and meaningful evaluation [81].

Third, our realist evaluation identified context-intervention configurations that specify which groups of recipients may be most likely to benefit from which intervention components. This enhances the utility of RCT findings by enriching the answer to “Does it work?” with insights into “Which components work and for whom?” [20,78]. Consequently, it offers greater pragmatic value for real-world decision-making by informing targeted intervention refinement and supporting strategy development, particularly important in health systems where resources are limited and tailoring is essential for effective scale-up.

This realist evaluation study, embedded within a broader KiT trial, was voluntary, and 10 (38%) of the 26 participants we contacted did not respond. Most of the interview participants who responded had high engagement with KiT prompts. This introduces potential selection bias and limits the transferability of the findings to individuals with moderate or low engagement levels. While highly engaged participants may experience positive outcomes [82], those with lower engagement may not benefit as much from the intervention in terms of improving their knowledge and motivation for T1D self-management. This may be due to the lack of a positive feedback loop that reinforces engagement and behavior change, potentially leading to an absence of the desired outcome. To better support adolescents and young adults, who may benefit from this intervention the most, future research may consider the following: (1) enhancing implementation strategies that target modifiable predictors of user engagement, such as incorporating the persuasive design and behavior change techniques to strengthen user motivation [83], timing interventions in summer months to reduce competing priorities [84], and fostering a supportive environment (particularly through health care providers [85]) and (2) exploring the specific needs of individuals with moderate or low engagement, recognizing that engagement is shaped by an interplay between intervention functions and user needs [86]. Second, given the exploratory and qualitative nature of this work, our primary goal was to generate early insights into the hypothesized mechanisms of change through which KiT might operate (ie, to validate or refine CIMO configurations). Quantitative evaluations of effectiveness were beyond the scope of this study and the available data. Combining these descriptive findings with the forthcoming KiT trial results will provide a more comprehensive understanding of the intervention’s impact. Third, although we recognize the value of understanding the representativeness of the interview subsample compared to the broader sample of all RCT intervention-arm participants, conducting a full comparison of engagement levels and outcome data was beyond the scope of this study. Finally, while we focused on individual-level contextual factors, we acknowledge the need to explore broader system-level factors (eg, health care access and social norms) and how they interact to influence emerging adults’ transition to adult care experiences.

The preliminary findings indicate that the KiT intervention is a promising intervention for supporting emerging adults living with T1D during their transition to adult care by enhancing their self-reported knowledge and motivation for self-management. T1D self-management information and suggestions were perceived as a universally valuable intervention component, suggesting it should remain a core component in interventions targeting this population. The KiT intervention also appears to operate through multiple pathways depending on specific users’ characteristics, supporting that digital health interventions should move beyond one-size-fits-all approaches to expect and allow users to tailor the intervention based on their needs, which is a competitive advantage that is enabled by their KiT’s digital construction. Future work that aims to support emerging adults living with T1D through behavioral interventions should focus on the unique needs of subgroups from the outset, including those who are recently diagnosed, have lower self-efficacy, or are experiencing diabetes distress, to tailor intervention components in line with individualized needs. This creates a user-friendly engagement environment more likely to enable sustained use over time. While improvements were observed in mechanisms and proximal outcomes (eg, knowledge and self-efficacy), changes in distal outcomes (eg, glycemic management) were less evident, necessitating the need for triangulation of these results from this study with the upcoming KiT trial findings.