PERMA Overall, our primary outcome variable, is a composite score from the PERMA-Profiler designed to measure self-reported flourishing in adults across 5 factors or subscales: Positive Emotion, Engagement, Relationships, Meaning, and Accomplishment [50]. It contains 15 items rated on an 11-point Likert scale (from 0 to 10). Scores are calculated as the average of the items comprising each factor or subscale, with higher scores indicating higher flourishing. A score of ≤6.4 is considered suboptimal, while a score of ≥8.0 is considered high [51]. It has demonstrated high internal consistency across settings (α=.92-.95) [52].

The PWS is a self-reported measure based on the Office of National Statistics 4 subjective well-being questions (ONS4) and thresholds. It contains 4 items, rated on a scale of 0 to 4 (0, “Disagree;” 1, “Neutral;” 2, “Agree;” and 3, “Strongly agree”). The summary score is calculated as an aggregate of the 4 items, transformed to a 0-100 scale, with higher scores indicating higher well-being. It shows good internal reliability (α=.90) [53].

The World Health Organization 5-item Well-being Index (WHO-5) is a self-reported measure of current mental well-being [54]. It consists of 5 statements rated on a 5-point scale in relation to the previous 2 weeks (5, “All of the time;” 4, “Most of the time;” 3, “More than half of the time;” 2, Less than half of the time;” 1, “Some of the time;” and 0, “At no time”). Scores are calculated by adding up each item and multiplying the total by 4, with 0 representing the worst imaginable well-being and 100 representing the best imaginable well-being. A clinical cutoff score of ≤50 indicates low well-being, while ≤28 indicates clinical depression [54]. It has high internal consistency and reliability (α=.81-.90), is valid in screening for depression, has strong construct validity among adolescents and adults, and is sensitive to change in interventions [54,55].

The PERMA Health subscale is a 3-item filler measure from the larger PERMA-Profiler, used to assess perceived physical health and vitality. Each item is rated from 0 to 10. Scores are calculated as the average of item scores, with higher scores indicating better perceived health. It is not a part of the main PERMA model and is instead used to provide additional context. It has high reliability across settings and contexts (α=.86-.93) [52].

The PERMA Positive Emotion subscale is a 3-item subscale from the PERMA-Profiler, designed to measure positive emotions, including joy, hope, pleasure, happiness, and contentment [50]. Each item is ranked from 0 to 10. Scores are calculated as the average of item scores, with higher scores indicating greater positivity. A score of ≤6.4 is considered suboptimal, while ≥8.0 is considered high [51]. It has demonstrated good internal consistency across settings (α=.71-.89) [52].

The PERMA Happiness subscale is a single filler item from the PERMA-Profiler, used alongside the primary PERMA model to provide additional context: “Taking all things together, how happy would you say that you are?” It is rated from 0 to 10, with higher scores indicating more happiness [52].

The PERMA Negative Emotion subscale is a 3-item filler measure from the PERMA-Profiler, used to assess tendencies toward feeling sad, anxious, or angry. Each item is rated from 0 to 10. Scores are calculated as the average of item scores. Lower scores are better (ie, less negativity). It is not a part of the main PERMA model and is instead used to provide additional context. It has acceptable reliability and has been shown to be negatively correlated with well-being measures (α=.63-.75) [52].

The Difficulties in Emotion Regulation Scale–Short Form (DERS-SF; emotion dysregulation) is an 18-item self-reported measure that assesses emotion dysregulation. It includes 6 subscales with 3 items each, including strategies, nonacceptance, impulse, goals, awareness, and clarity. Each item is rated on a 5-point scale of frequency ranging from 1 (“Almost never 0-10%”) to 5 (“Almost always 91-100%”). Items are summed (awareness items are reverse coded) to create a total score of 18 to 90, with higher scores indicating greater difficulties with emotion regulation [56]. It is a validated measure for assessing emotion regulation problems and possesses high internal consistency (α=.89-.91) [57].

Patient Health Questionnaire-8 (PHQ-8) is an 8-item diagnostic and severity measure for assessing depressive disorders in large clinical settings [58]. Statements about the previous 2 weeks are rated on a 4-point scale ranging from 0 (“Not at all”) to 3 (“Nearly every day”). Scores are reported on a scale from 0 to 24, with higher scores indicating greater severity in depressive symptoms. Scores above 10 indicate clinically significant depressive symptoms. It has high internal consistency, reliability (α=.82), and validity, with 96.5% of those who score above the clinical cutoff having a depression diagnosis [58,59].

Generalized Anxiety Disorder-7 (GAD-7) consists of 7 items rated on a 4-point Likert scale ranging from 0 (“Not at all”) to 3 (“Nearly every day”), with the same options as in the PHQ-8 [60]. Items are summed to create a total severity score ranging from 0 to 21, with higher scores indicating greater symptom severity. Scores above 10 indicate clinically significant anxiety symptoms. It has high internal consistency, reliability (α=.89), and validity, with moderate correlations with related constructs such as depression and self-esteem [61].

The PERMA Relationships subscale consists of 3 items from the larger PERMA-Profiler that measure whether a person feels cared about, authentically connected, and securely connected to others [50]. Each item is rated on a scale from 0 to 10. Scores are calculated as the average of item scores, with higher scores indicating higher social connectedness. A score of ≤6.4 is considered suboptimal, while ≥8.0 is considered high [51]. It has demonstrated high internal consistency across settings (α=.75-.85) [52].

The PERMA Loneliness subscale is a single filler item from the PERMA-Profiler, used alongside the primary PERMA model to provide additional context: “How lonely do you feel in your daily life?” It is rated from 0 to 10, with lower scores indicating less loneliness. It is negatively correlated with the PERMA well-being measures [52].

The PERMA Accomplishment subscale contains 3 items from the PERMA-Profiler that measure drive toward personal goals [50]. Each item is rated from 0 to 10. Scores are calculated as the average of item scores, with higher scores indicating higher accomplishment. A score of ≤6.4 is considered suboptimal, while ≥8.0 is considered high [51]. It has demonstrated good internal consistency across settings (α=.70-.86) [52].

The PERMA Meaning subscale contains 3 items from the PERMA-Profiler that measure sense of purpose [50]. Each item is rated from 0 to 10. Scores are calculated as the average of item scores, with higher scores indicating a greater sense of meaning. A score of ≤6.4 is considered suboptimal, while ≥8.0 is considered high [51]. It has demonstrated high internal consistency across settings (α=.85-.92) [52].

The PERMA Engagement subscale contains 3 items from the PERMA-Profiler that measure absorption, interest, and involvement in the world [50]. Each item is rated from 0 to 10. Scores are calculated as the average of item scores, with higher scores indicating greater engagement. A score of ≤6.4 is considered suboptimal, while ≥8.0 is considered high [51]. It has acceptable internal consistency (α=.60-.81) [52].

App engagement is assessed by the total and weekly sums of activities completed in the app during the trial period, calculated as the sum of lessons, posts, replies, likes, and mood surveys.

Weekly sums are assessed relative to recommended benchmarks from GUIDE (≥3 lessons, posts, replies, likes, and mood surveys), as well as minimum usage standards (≥3 activities).

The Acceptability of Intervention Measure (AIM) assesses how agreeable or likable an intervention is to stakeholders. It contains 4 items rated on a 5-point Likert Scale ranging from 1 (“Completely disagree”) to 5 (“Completely agree”). Scores are calculated as the average of item scores, with higher scores indicating greater acceptability. It has been shown to have good internal consistency and test-retest reliability (α=.83-.85) [62].

The Intervention Appropriateness Measure (IAM) assesses the perceived fit of an intervention to a specific setting. It contains 4 items rated on a 5-point Likert Scale ranging from 1 (“Completely disagree”) to 5 (“Completely agree”). Scores are calculated as the average of item scores, with higher scores indicating greater appropriateness. It has been shown to have high internal consistency and test-retest reliability (α=.87-.91) [62].

The Feasibility of Intervention Measure (FIM) assesses how easily an intervention can be carried out within a given setting. It contains 4 items rated on a 5-point Likert Scale ranging from 1 (“Completely disagree”) to 5 (“Completely agree”). Scores are calculated as the average of item scores, with higher scores indicating greater feasibility. It has been shown to have good internal consistency and test-retest reliability (α=.88-.89) [62].

The System Usability Scale (SUS) assesses the perceptions of usability of a given system (in this case, GUIDE) [63]. It contains 10 items rated on a 5-point Likert Scale ranging from 1 (“Strongly disagree”) to 5 (“Strongly agree”). Responses are transformed to get a total range of possible values from 0 to 100, with higher scores indicating better usability. It has high internal consistency and reliability (α=.91) [64]. Scores above 68 are regarded as satisfactory or above average, while scores above 80 are considered good [65].

For our primary outcome variable, PERMA Overall, the main effect of time was significant and had a moderate effect size (F_1,104=7.78; P=.006; η_p²=0.07). The time×group interaction was not significant (F_2,104=1.15; P=.32; η_p²=0.02). Post hoc pairwise comparisons revealed that the GUIDE-only group experienced a significantly greater improvement in PERMA Overall when compared to the waitlist control group (raw mean=0.11; P=.03; d=0.31), a difference that translates to a 1% greater percentile point increase in the raw PERMA Overall score or a 9.1% greater increase when controlling for time. Raw scores for all 3 groups fell in the moderate well-being range for all time points (Figure 3A).

For the PWS, the main effect of time was significant and had a large effect size (F_1,103=19.22; P<.001; η_p²=0.16) when controlling for the baseline score. The time×baseline interaction effect was also significant and had a large effect size (F_2,103=17.19; P<.001; η_p²=0.14). The time×group interaction was not significant (P=.35). See Figure 3B for raw scores by treatment group over time.

For the WHO-5, the main effect of time was significant and had a moderate effect size (F_1,104=9.09; P=.003; η_p²=0.08). The time×group interaction was not significant (P=.31). In post hoc pairwise comparisons, there was a borderline significant mean difference (MD) between the GUIDE-only and waitlist control groups, with the GUIDE-only group on average improving more (raw MD=5.03; P=.07; d=0.27), a difference that translates to a 5.04% greater percentile point increase in the raw score or a 10.25% greater increase when controlling for time. The raw mean scores were in the moderate well-being range for the treatment groups across time (Figure 3C).

For perceived health, the main effect of time was significant and had a large effect size (F_1,103=19.48; P<.001; η_p²=0.16). The time×baseline interaction was significant and had a large effect size (F_2,103=14.03; P<.001; η_p²=0.12). The time×group interaction was not significant (P=.12). In post hoc pairwise comparisons, there were no significant differences (all P≥.05). See Figure 3D for raw mean scores by group over time.

For PERMA Positive Emotion, the main effect of time was significant and had a moderate effect size (F_1,104=11.88; P<.001; η_p²=0.10). The time×group interaction was not significant (P=.33). In post hoc pairwise comparisons, the GUIDE-only group showed a significant improvement compared to the waitlist control group (raw MD=0.29; P=.03; d=0.31), translating to a 10.9% increase when controlling for time. See Figure 4A for raw mean scores by group over time.

For PERMA Happiness, the main effect of time was significant and had a small effect size (F_1,104=4.14; P=.04; η_p²=0.04). The time×group interaction was not significant (P=.12). In post hoc pairwise comparisons, the GUIDE-only group showed significant improvement in this item relative to the waitlist control group (raw MD=1.24; P=.04; d=0.30), translating to a 12.4% greater percentile point improvement or a 10.6% greater improvement when controlling for time. See Figure 4B for raw scores by group over time.

For PERMA Negative Emotion, the main effect of time was significant and had a large effect size (F_1,104=18.02; P<.001; η_p²=0.16). The time×group interaction was not significant (P=.48). All 3 treatment groups showed a decrease in negative emotion over time (Figure 4C).

For emotion dysregulation (DERS-SF), the main effect of time was significant and had a moderate effect size (F_1,103=13.14; P<.001; η_p²=0.11) when controlling for baseline. The time×baseline interaction was also significant and had a large effect size (F_2,103=20.31; P<.001; η_p²=0.17). The time×group interaction was borderline significant and had a moderate effect size (F_2,103=2.88; P=.06; η_p²=0.05). In group×time post hoc pairwise comparisons, only the GUIDE+incentives group showed a significant reduction in emotion dysregulation over time, with a moderate effect size (raw MD=–4.56; P<.001; η_p²=0.09), indicating that a 6.33% greater percentile point reduction in raw emotion dysregulation for this group drove the borderline significant time×group interaction effect. Post hoc pairwise comparisons were not significant (all P≥.05). See Figure 4D for raw scores by treatment group over time.

For depression symptoms (PHQ-8), the main effect of time was significant and had a large effect size (F_1,104=16.99; P<.001; η_p²=0.14). The time×group interaction was not significant (P=.64). In post hoc pairwise comparisons, the GUIDE-only group showed a significantly greater reduction in depression symptoms compared to the waitlist control group (raw mean=–1.34; P=.04; d=–0.30), translating to a 5.58% greater percentile point reduction in the raw depression score for the GUIDE-only group. As shown in Figure 5A, the average depression score for all groups fell below clinical levels for both time periods; however, a substantial portion of our participants did meet the clinical criteria: 29.0% (31/107) at baseline and 16.8% (18/107) at posttrial. A majority of participants (68/115, 59.1%) had symptoms of subclinical or clinical depression (≥5) at baseline, while nearly half (53/107, 49.5%) had these symptoms at posttrial. See Figure S2 in Multimedia Appendix 1 for participant depression severity across time.

For anxiety symptoms (GAD-7), the main effect of time was significant and had a large effect size when controlling for baseline (F_1,103=14.33; P<.001; η_p²=0.12). The time×baseline interaction was also significant and had a larger effect size (F_2,103=23.92; P<.001; η_p²=0.19). The time×group interaction was not significant (P=.29). In post hoc pairwise comparisons, the GUIDE-only group showed a significant relative reduction in anxiety compared to the waitlist control group (raw MD=–0.5; P=.05; d=–0.29). Mean raw anxiety scores exceeded the clinical cutoff for all 3 groups over time (Figure 5B), indicating a high level of anxiety present in our sample. A majority of participants met the criteria for clinical anxiety at both baseline (82/115, 71.3%) and posttrial (70/107, 65.4%), and 100% of participants met the criteria for mild anxiety at both time periods. See Figure S3 of Multimedia Appendix 1 for participant anxiety level across time.

For PERMA Relationships, the main effect of time was significant and had a large effect size when controlling for baseline (F_1,103=14.86; P<.001; η_p²=0.13). The time×baseline interaction was also significant and had a moderate effect size (F_1,103=11.40; P=.001; η_p²=0.10). The time×group interaction was not significant (P=.12). Post hoc pairwise comparisons were not significant (all P≥.05). See Figure 6A for raw mean scores by treatment group over time.

For PERMA Loneliness, the main effect of time was significant and had a moderate effect size (F_1,103=7.55; P=.007; η_p²=0.07). The time×baseline interaction was significant and had a large effect size (F_1,103=26.78; P<.001; η_p²=0.21). The time×group interaction was not significant (P=.22). See Figure 6B for raw mean scores by treatment group over time.

For PERMA Accomplishment, the main effect of time was significant and had a large effect size when controlling for baseline (F_1,103=19.16; P<.001; η_p²=0.16). The time×baseline interaction was also significant and had a large effect size (F_2,103=16.00; P<.001; η_p²=0.13). The time×group interaction was not significant (P=.69). Post hoc pairwise comparisons were not significant (all P≥.05). See Figure 7A for raw mean scores by treatment group over time.

For PERMA Meaning, the main effect of time was significant and had a moderate effect size when controlling for baseline (F_1,103=9.33; P=.003; η_p²=0.08). The time×baseline interaction was significant and had a moderate effect size (F_2,103=7.86; P=.006; η_p²=0.07). The time×group interaction was not significant (P=.29). Post hoc pairwise comparisons were not significant (all P≥.05). See Figure 7B for raw mean scores by treatment group over time.

For PERMA Engagement, neither time (P=.26) nor time×group (P=.68) was significant. Mean scores for all 3 groups across both time periods fell near the suboptimal cutoff (Figure 7C).

We explored outcomes for active first responders (n=93). See Table S2 in Multimedia Appendix 1 for a breakdown by responder type. We reran our repeated measures ANOVAs for select outcome measures using data from active first responders only. We excluded “filler” items from the PERMA-Profiler (ie, Negative Emotion, Happiness, Health, and Loneliness), as well as measures without significant differences in our main analysis (ie, Engagement). See Table 4 for a summary of our findings.

Time was significant and had moderate to large effect sizes for all outcome measures (P<.05), except PERMA Overall which was borderline significant and showed a small effect size (P=.08). There were no significant time×group interactions; however, 2 interactions were borderline significant and had moderate effect sizes: emotion dysregulation (DERS-SF; F_2,82=2.35; P=.10; η_p²=0.05) and depression symptoms (PHQ-8; F_2,82=2.68; P=.07; η_p²=0.06). In group×time pairwise comparisons for these outcome measures, the GUIDE+incentives group showed the most significant improvement, with a 4.63% reduction in emotion dysregulation (P=.03) and a 9.00% reduction in depression symptoms (P=.001) based on adjusted means and showing moderate and large effect sizes. Depression symptoms decreased by 5.13% for the waitlist control group (P=.03), but the effect sizes and MDs were half as large as those in the GUIDE+incentives group.

In pairwise comparisons by group, the GUIDE+incentives group performed better than the waitlist control group for all outcomes, but the MDs between these groups were not significant (all P≥.05). The GUIDE-only group improved more than the waitlist control group across wellness, emotional well-being, mental health, and social connectedness measures; however, the difference between groups was only significant for wellness (PERMA Overall: P=.03) and borderline significant for emotional well-being (PERMA Positive Emotion: P=.08). The GUIDE-only group did not improve significantly over the waitlist control group on any personal growth measure (all P≥.05). When comparing the GUIDE+incentives and GUIDE-only groups, the results were mixed, indicating that half of our outcomes improved more for participants who received financial incentives, while the other half improved more for participants who did not receive the incentives, although none of the differences between the 2 interventions were statistically significant (all P≥.05). The GUIDE+incentives group showed greater improvement in personal growth measures, while the GUIDE-only group showed greater improvement in wellness and emotional well-being measures.

During our intake meetings, many participants disclosed that they were leaders in their organization or involved in wellness for their community. We did not include an item to capture this in our demographic survey, so we selected participants with less education (college or less, n=63) and fewer years of employment (<15 years, n=66) as proxies for less leadership or wellness experience in a sensitivity analysis. See Tables S3 and S4 in Multimedia Appendix 1 for our findings.

While these sensitivity analyses were not powered, pairwise comparisons of less experienced first responders suggested that GUIDE with financial incentives may have led to reductions in emotion dysregulation (P=.009), depression (P=.002), and anxiety (P=.04). However, these differences were not significant when comparing group pre-post deltas (all P≥.05). Among first responders with less education, the GUIDE-only group showed borderline significant improvements compared to the waitlist control group for wellness (P=.06) and emotional well-being (P=.08) in pairwise group comparisons.

To explore outcomes for participants with military affiliation, we conducted repeated measures ANOVAs for veterans and active-duty military personnel (n=45). We excluded “filler” items from the PERMA-Profiler, as well as the PERMA Engagement subscale, as in the responder analysis. The findings are presented in Table 5.

The main effect of time was significant for all wellness (P<.01), emotional well-being (P<.05), mental health (P<.001), social connectedness (P<.001), and personal growth (P<.05) measures, except for PERMA Overall, which was borderline significant (P=.10).

Unlike our main analysis, 2 measures had significant time×group interactions and showed large effect sizes: GAD-7 for anxiety (F_2,40=3.85; P=.03; η_p²=0.16) and PERMA Relationships (F_2,40=3.85; P=.03; η_p²=0.16).

In post hoc group×time pairwise comparisons of depression symptoms, the GUIDE+incentives group showed a significant improvement over time, with a large effect size (raw MD=–1.44; P=.001; η_p²=0.25), translating to a 6.86% greater percentile point reduction in the raw anxiety score or a 13.76% greater reduction when controlling for baseline. In post hoc between-group pairwise comparisons, the GUIDE-only group showed a significantly greater reduction in anxiety symptoms compared to the waitlist control group (raw MD=–1.69; P=.03; d=–0.31), a difference that translates to an 8.05% greater percentile point reduction in the raw anxiety score or a 7.62% greater reduction when controlling for baseline.

For PERMA Relationships, post hoc group×time pairwise comparisons showed that both the GUIDE+incentives group (raw MD=0.81, SD 0.76; P=.02) and GUIDE-only group (raw MD=0.33; P=.05; η_p²=0.13) showed significant improvements over time, with large and moderate effect sizes, respectively, while the waitlist control group (raw MD=–0.22; P=.02; η_p²=0.09) did not show such an improvement. In post hoc between-group pairwise comparisons, both the GUIDE+incentives (P=.04) and GUIDE-only (P=.08) groups showed significantly greater improvements on this measure when compared to the waitlist control group. This indicates that social connectedness improved for both groups with GUIDE access but did not improve in the waitlist control group.

Beyond anxiety and social connectedness, the GUIDE-only group showed significant improvements over the waitlist control group in wellness (WHO-5; P=.049), emotional well-being (PERMA Positive Emotion; P=.02), and mental health (PHQ-8 for depression; P=.051), although the effect sizes were small. Wellness, as measured by PERMA Overall, was borderline significant in the GUIDE-only group compared with the waitlist control group (P=.09). The GUIDE-only group also had significantly reduced depression symptoms compared to the GUIDE+incentives group (P=.047) and borderline significant relative improvement in well-being (WHO-5; P=.07). The GUIDE+incentives group showed a significant improvement over the waitlist control group for social connectedness (P=.04). Together, these results suggest that GUIDE alone may perform better without financial incentives for military personnel and veterans.

In our analysis of participants without military affiliation (ie, first responders only), none of the time×group interactions were significant (all P≥.05). Details are provided in Table S5 in Multimedia Appendix 1.

To explore how sex influenced participant outcomes, we conducted exploratory analyses of select outcome measures by rerunning our repeated measures ANOVAs for male (n=76) and female (n=31) participants, since previous research has shown significant variation between sexes on self-report measures related to health and mental health [74]. The results for male participants were powered, and the findings are reported in Table 6.

The main effect of time was significant across all wellness (P<.05), emotional well-being (P<.05), mental health (P<.001), personal growth (P<.01), and social connectedness (P<.001) measures included in the analysis. For PERMA Relationships, the time×group interaction was significant (P=.04) and had a moderate effect size when controlling for baseline. In post hoc group×time pairwise comparisons, only the GUIDE+incentives group showed a significant improvement (P=.005). Post hoc pairwise comparisons by group corroborated these findings and indicated that the GUIDE+incentives group performed significantly better than the waitlist control group on this measure (P=.03). Altogether, these findings indicate that male participants may feel more socially connected when using GUIDE with financial incentives, rather than without.

For emotion dysregulation (DERS-SF), the time×group interaction was borderline significant (P=.10). In post hoc group×time pairwise comparisons, only the GUIDE+incentives group showed a significant reduction in emotion dysregulation over time when controlling for baseline (P=.006), although in post hoc pairwise comparisons by group, there were no significant differences (all P≥.05). GUIDE with financial incentives may improve emotion regulation for male users, but these findings are not as strong as for PERMA Relationships.

The results for females are reported in Table S6 in Multimedia Appendix 1. The main effect of time was only significant for the included emotional well-being (P<.01) and mental health (P<.05) measures. None of the time×group interactions were significant (all P≥.05), but unlike the male sample, these analyses were not powered and thus should be interpreted cautiously. In post hoc pairwise comparisons by group, we found that the GUIDE-only group showed a significant reduction in depression symptoms (PHQ-8) relative to the waitlist control group (P=.02), indicating that GUIDE usage without financial incentives may have helped reduce these symptoms relative to the control group. There was also a borderline significant difference between the GUIDE-only and waitlist control groups for PERMA Positive Emotion in post hoc pairwise comparisons (P=.08).

In summary, the effect of GUIDE and financial incentives on wellness, emotional well-being, and mental health may differ by sex. Male participants benefited more from the GUIDE+incentives condition, while female participants benefited from the GUIDE-only condition, indicating that males may be more motivated by financial incentives than females.

Like in our main analysis, the main effect of time for active first responders was significant and had moderate to large effect sizes for all reported outcome measures, except PERMA Overall, which was borderline significant and had a small effect size. There were no significant time×group interactions; however, the interaction effects for emotion dysregulation and depression symptoms were borderline significant. The GUIDE+incentives group showed the most significant improvement in these measures, with a 4.63% greater percentile point reduction in emotion dysregulation and 9.00% greater percentile point reduction in anxiety symptoms across the 2 time periods based on adjusted means, with moderate and large effect sizes, respectively. Depression symptoms also decreased by 5.13% in the waitlist control group, but the effect sizes and MDs were half as large as those in the GUIDE+incentives group.

In between-group comparisons, the GUIDE+incentives group performed better than the waitlist control group for all outcomes, but the MDs between these groups were not significant. The GUIDE-only group improved more than the waitlist control group across wellness, emotional well-being, mental health, and social connectedness measures; however, the difference between groups was only significant for wellness (PERMA Overall) and borderline significant for emotional well-being (Positive Emotion). Notably, the GUIDE-only group did not improve significantly over the waitlist control group in personal growth measures. When comparing the GUIDE+incentives and GUIDE-only groups, the results were mixed, with half of the outcomes improving more for responders who received financial incentives and the other half improving more without incentives, although the differences were not statistically significant, indicating that financial incentives are not integral to efficacy.

In sensitivity analyses selecting for first responders with less education and experience, group×time pairwise comparisons showed that the GUIDE+incentives group had more significant improvements and larger effect sizes compared to the findings in the analysis of all responders. While these subgroup sensitivity analyses were not powered, they imply that financial incentives may act as a more powerful motivator for first responders with less education and experience, particularly for reducing emotion dysregulation, depression, and anxiety. In terms of wellness and emotional well-being, however, the GUIDE-only approach still performed significantly better, indicating that these outcomes may be less influenced by financial incentives, regardless of experience or education.

Veterans and active-duty military personnel who used GUIDE showed a greater improvement in outcome measures over time when compared to participants without military affiliation. These improvements were more significant and had larger effect sizes than those for participants overall, indicating that GUIDE may be more effective in improving wellness, emotional well-being, mental health, and social connectedness for veterans and active-duty military personnel. In between-group post hoc pairwise comparisons, the GUIDE-only group showed significant improvements over time compared to the waitlist control group for wellness (WHO-5), emotional well-being (PERMA Positive Emotion), mental health (GAD-7 anxiety and PHQ-8 depression symptoms), and social connectedness (PERMA Relationships), while the GUIDE+incentives group showed a significant improvement over the waitlist control group for only social connectedness (PERMA Relationships). Of note, the GUIDE-only group showed a significant and borderline significant improvement in PHQ-8 depression symptoms and WHO-5 well-being, respectively, when compared to the GUIDE+incentives group, strengthening the case that GUIDE alone leads to better wellness outcomes than GUIDE with financial incentives for military-affiliated participants. These findings are extremely encouraging and suggest that GUIDE may be just as, if not more, effective than existing mental health-related apps created by the Veterans Affairs (VA) or Department of Defense (DoD) [84]. While these subgroup results are promising, they are underpowered and must be interpreted with caution. Future research should include a larger number of veterans and military personnel to validate our findings.

Our analyses by sex revealed that GUIDE may be an especially good fit for males, which is promising since males make up about 75% of first responders [85], 82.5% of active-duty military personnel [86], and 89% of veterans [87]. Additionally, males are more likely to commit suicide [88] and are less likely to seek help or avoid treatment due to stigma [89].

Male participants allocated to the GUIDE+incentives group improved significantly over the waitlist control group in terms of social connectedness (PERMA Relationships) and had the most significant improvements over time for our selected outcome measures (PERMA Overall, WHO-5, PERMA Positive Emotion, DERS-SF [emotion dysregulation], depression [PHQ-8], anxiety [GAD-7], PERMA Meaning, and PERMA Relationships), indicating that they may have responded more to financial incentives than females, a finding that aligns with previous research [90]. This suggests that incorporating financial incentives may yield more positive outcomes and app engagement for male GUIDE users.

Overall, 67% (51/76) of participants met our minimum usage standards for all weeks, while only 8% (6/76) of participants met the recommended benchmarks for all weeks. This indicates that the recommended benchmarks may be high for this population, since individuals willing to volunteer for a clinical trial are likely more motivated to persist than average.

The app attrition rate of 13% (n=10) was higher than the trial attrition rate of 6% (n=7), both of which were substantially lower than the weighted meta-analytic attrition rate of 24.7% from a recent study of mindfulness apps, a promising sign about the uptake of GUIDE among first responders and military personnel [91]. Attrition was higher in the GUIDE intervention groups than in the waitlist control group, which is typical for digital mental health interventions, and was the highest in the GUIDE+incentives group, indicating that app usage and financial incentives may have been considered burdensome by some participants.

Financial incentives significantly increased the average number of posts and replies in the small group chat of GUIDE. This suggests that financial incentives may drive engagement with the peer-support element of GUIDE or that the peer support feature has room to improve on usability or acceptability to keep users engaged. Financial incentives did not lead to significantly more lessons completed, indicating that the education feature keeps users engaged and that they are motivated to complete these lessons with or without the opportunity for compensation.

When assessing how app engagement (overall and by feature) was correlated with outcome measures, lessons, posts, and replies were associated with improvements in wellness and emotional well-being measures. Additionally, all features were associated with an improvement in accomplishment, indicating that participants who engaged with the app more felt more accomplished, regardless of the features used. Replies and likes, indicative of peer-to-peer interaction, were significantly correlated with a reduction in depression symptoms, indicating that these interactions may be beneficial for mental health. This finding is important to keep in mind, since the number of replies per participant was highly dependent on whether they received financial incentives. Figuring out a way to increase engagement and interaction between users may lead to an improvement across these outcomes, especially for mental health measures. Financial incentives may be an effective way to encourage use for some participants.

While financial incentives effectively increased app engagement, particularly in small group chats, they did not lead to corresponding improvements in primary outcomes. This suggests that intrinsic motivation may be essential for first responders, veterans, and military personnel to benefit from an app like GUIDE. According to the self-determination theory, individuals respond differently to financial incentives based on their intrinsic values and beliefs. Because financial materialism is a less reliable motivator than financial altruism, especially in service-oriented professions, gamification strategies that foster intrinsic motivation may be more effective in driving sustained app engagement [92]. This explanation is supported by our dropout analysis, which found that participants with no prior behavioral health care experience and lower app usage during the trial (ie, those less intrinsically motivated to engage with mental health interventions) were significantly less likely to complete the posttrial assessment, regardless of financial incentives.

Appropriateness and feasibility, which represent the suitability of an intervention for a given population and the practicality of implementation in real-world conditions, were good according to our participants. This indicates that GUIDE is a good fit for first responders and military personnel and that it would be reasonable to implement given available resources.

Acceptability, which represents the degree to which an intervention is viewed positively, was satisfactory. High acceptability is important for digital mental health interventions, since it has been identified as a factor that increases both uptake and adherence among individuals with depression and anxiety [93]. Exploring opportunities to increase acceptability may improve outcomes for GUIDE members.

Usability was above average for GUIDE, but the scores indicate that there is room for improvement in this measure as well. Usability is critical when designing an effective digital mental health intervention, since users are more likely to stop using an app due to usability issues (eg, bugs, poor user interaction design, data loss, battery and memory usage, and lack of explanation or guidance) [94]. Usability scores for this trial may have been impacted by app outages experienced during the second half of the trial.

Overall, GUIDE performed above average across implementation outcomes and technical merit. Focusing on acceptability and usability can help take this app to the next level, which in turn could help drive adherence, engagement, and, ultimately, further improvement in wellness and mental health outcomes.