Experiences of sexual assault or harassment during military service, known as military sexual trauma (MST) [1], are common and can have lasting negative effects on military service members and veterans, including impacts on military preparedness, postservice readjustment, and long-term mental and physical health. In a national survey of 20,563 recent veterans conducted from 2009 to 2011 by the US Department of Veterans Affairs (VA), approximately 41% of women and 4% of men reported experiences of MST [2]. However, the actual prevalence of MST may be even higher (eg, as high as 74% in women veterans [3]) due to underreporting and the substantial heterogeneity in definitions and methods for assessing MST across studies [4,5].

MST can have profound mental health implications, often leading to severe psychological distress, poor psychosocial functioning, and reduced quality of life. The most common mental health condition associated with MST is posttraumatic stress disorder (PTSD) [5]. Military personnel are at high risk for developing PTSD due to combat and other traumatic exposures experienced during military service, and experiencing MST further exacerbates this risk [6]. In addition, experiences of MST are associated with other mental health conditions such as substance use [7], depression, and suicidality [8], all of which compound the psychological burden on affected individuals. These findings underscore the critical need for interventions that support individuals coping with the aftereffects of MST.

MST, like other forms of trauma, can also lead to increased dysfunctional beliefs or maladaptive thinking patterns. Research suggests that negative posttraumatic cognitions (NPCs, ie, distorted or maladaptive thoughts and beliefs that develop following trauma) play a crucial role in the mechanisms of action of PTSD [9,10]. Specifically, trauma can disrupt an individual’s core beliefs about themselves, others, or the world, which can lead to maladaptive beliefs (eg, “I don’t feel safe anywhere anymore” [11]) and loss of meaning in life [9,12]. Even when controlling for other posttraumatic risk factors, NPCs can intensify other PTSD symptoms such as avoidance and hypervigilance [10]. Overall, NPCs can hinder recovery after traumatic events and may be especially problematic following experiences of MST. Notably, MST is frequently associated with feelings of self-stigma, or the internalization of negative stereotypes that can lead to guilt, shame, and self-blame. This can manifest in NPCs such as, “The event happened to me because of the sort of person I am,” or “Somebody else would not have gotten into this situation” [13]. The distinctive nature of the experience of MST highlights the importance of tailored, evidence-based approaches that address NPCs and other symptoms of sexual trauma.

Although numerous psychotherapy interventions exist to address symptoms of PTSD following traumatic experiences such as MST (eg, Cognitive Processing Therapy, Prolonged Exposure, and Acceptance and Commitment Therapy) [14], individuals who have experienced sexual trauma face significant barriers to help-seeking. Societal stigma around sexual trauma can translate into fear of judgment in survivors, and victim-blaming and disbelief from others. For example, one study found that higher self-stigma is associated with a lower likelihood of disclosing MST in health care screenings [15]. In addition, the setting in which MST occurs (ie, service members living in close quarters on a long-term basis; hierarchical rank structures) can produce fears around subsequent violence, work retaliation, and ostracism by colleagues [16]. Consequently, negative institutional experiences may prompt confidentiality concerns, feelings of distrust toward providers, and discomfort disclosing MST experiences in health care settings [17-19]. These kinds of barriers to help-seeking, nondisclosure with providers, and underuse of health care services can reduce an individual’s opportunities for recovery.

Fortunately, widely accessible technology-based solutions, such as mental health apps, have the potential to expand access to psychoeducation, coping tools, and resources for self-management purposes [20]. They can also bolster patient autonomy, offer 24/7 support when traditional care is unavailable, and preserve anonymity to mitigate stigma—benefits that prove especially valuable for individuals coping with sensitive traumatic events.

Increasingly, researchers have demonstrated the effectiveness of mental health apps in improving coping skills, reducing mental health symptoms, and enhancing users’ sense of well-being and quality of life across populations and mental health challenges [21]. The PTSD Coach app, developed by the US Department of Veterans Affairs (VA) National Center for PTSD, is one well-researched example that has demonstrated feasibility, acceptability, and effectiveness across 16 studies [22]. Digital mental health technologies for addressing specific trauma types, such as sexual trauma and MST [23,24], have also emerged. For example, apps for sexual violence (especially intimate partner violence) are widely available, but typically serve informational, preventative, or peer support purposes [25-27].

The Beyond MST app (US Department of Veterans Affairs [VA] National Center for PTSD) is uniquely designed to help users self-manage MST-related psychosocial challenges (eg, mental health symptoms, behavioral issues, and relationship challenges), enhance their general well-being, and find hope via evidence-based psychoeducation and coping tools [28,29]. To date, no prior research on digital mental health technologies for individuals who experienced MST, including Beyond MST, has been conducted. While usability and effectiveness studies are important, research is also needed to identify factors that may limit users’ engagement with app content and consequently impact their effectiveness [30].

Low levels of user engagement with mental health technologies in real-world, nonresearch settings are a significant problem, as many users discontinue use after only a few interactions. For instance, a review of 93 popular mental health apps revealed a median 15-day user retention rate of just 3.9% (IQR 10.3%) [31]. This pattern mirrors the issue of psychotherapy dropout, which can impede patient recovery; similarly, insufficient engagement may hinder users from achieving their full therapeutic benefit.

While early research on digital mental health technology engagement primarily examined its link to health outcomes, recent efforts have shifted toward uncovering the underlying factors that impede or promote user engagement. These include setting-, intervention-, and user-related predictors. For instance, a large body of research has examined intervention-level factors such as the addition of human coaching, gamification, and persuasive technology [30]. However, user-level predictors—particularly symptom severity—remain underexplored.

A systematic review of 208 studies on digital mental health interventions found that only 19 studies examined the relationship between symptom severity and objective measures of user engagement. Most indicated that higher symptom severity, especially related to depression and fatigue, was linked to lower engagement [32]. Yet, findings are mixed: another review reported that while 2 studies associated greater symptom severity with increased usage, 3 found the opposite [30]. Conflicting results may be due to the range of mental health conditions (eg, depression, anxiety, bipolar disorder, and schizophrenia) and digital interventions (eg, web-based services, telehealth programs, and mobile apps) assessed, as well as diversity in the engagement measures used.

The relationship between symptom severity and mental health app engagement may be further complicated by potential curvilinear patterns [30], yet very few studies have modeled symptom severity as a continuous predictor that includes quadratic (or higher-order) terms, leaving such nonlinear relationships largely untested. There is some evidence of possible nonlinear relationships between depression symptoms and mental health app engagement, with moderate symptom levels associated with relatively higher engagement, and mild and severe symptom levels associated with relatively lower engagement (eg, [33]). Possible inverted-U patterns may reflect varying barriers across symptom levels—for instance, low perceived need among users with mild symptoms and increased cognitive burden among users with severe symptoms.

Furthermore, research exploring whether posttraumatic symptom severity predicts engagement with digital mental health technologies remains limited. While some evidence suggests that higher PTSD symptoms are associated with greater intentions to use such technologies, self-reported intentions may not translate into actual behavior (eg, [32,34]). This underscores the importance of analyzing objective engagement metrics. For instance, usage data from COVID Coach (US Department of Veterans Affairs [VA] National Center for PTSD)—an app offering coping tools and resources during the COVID-19 pandemic—revealed that users with significant PTSD symptoms were more likely to use the app for more days on average than those with subthreshold symptoms [33]. Taken together, the relationship between symptom severity and engagement with digital health technologies is unclear and may vary based on symptom type and engagement measures used. Further research is needed to delineate the relationships between trauma-related symptom measures and engagement with trauma-focused apps, especially following experiences of MST where no research has been conducted.

This retrospective observational study is the first to evaluate a mental health app specifically designed for veterans and service members who have experienced MST. Given the limited research in this area, an exploratory approach was taken to (1) characterize trauma-related symptom levels and app engagement among subsamples of users who completed in-app assessments, and (2) for those subsamples, examine the relationship between trauma-related measures and objective app engagement measures. Usage data from a large sample of Beyond MST app users were processed and analyzed. Two primary predictors were examined: PTSD symptom severity and the severity of NPCs. Mental well-being was included as a secondary predictor to serve as a comparator. App engagement was measured via 3 types of variables: measures of duration of app use, frequencies of feature access, and frequency of feature use. Overall, findings may enhance our understanding of trauma-related predictors of digital mental health engagement among real-world users coping with MST.

The Beyond MST app was developed by the VA National Center for PTSD in collaboration with the national VA MST Support Team [28,29]. It is also part of a larger portfolio of mental health apps developed and maintained by the National Center for PTSD [35]. Beyond MST can be used for self-care purposes (eg, to self-manage MST-related symptoms) or in conjunction with professional health care. However, the app is not intended to replace professional therapy or treatment. While app content was curated for individuals who experienced MST, it may also be helpful to those who have experienced other types of sexual trauma outside of military settings (eg, intimate partner violence).

Beyond MST is publicly available and can be downloaded for free from the Google Play and Apple App stores on smartphones or tablets. It was first released in March 2021 and has since undergone ongoing maintenance and app updates. Once downloaded, the app can be used without an internet connection. Additionally, no user account is required. The app is also private and secure, meaning all usage data collected are anonymous, and users can opt out of providing usage data.

Users can access the following four content areas: (1) the “Home” screen with inspirational quotes, user-selected bookmarks, and a programmed conversational feature for app guidance, (2) “Tools” for developing and practicing active coping skills, (3) “Learn” for psychoeducation topics and resources (eg, Veterans Crisis Line), and (4) “Progress” for tracking progress over time through in-app assessments and a goal-setting feature (Figure 1). The Tools section includes 54 coping tools (eg, “Managing Reactions” and “Finding My Values”). The Learn section includes 49 primary psychoeducation topics (eg, “Self-Blame and Anger” and “Relationships and Health”), and 29 subtopics embedded within the primary topics (eg, Types of Support). Features were designed based on Cognitive Behavioral Therapy (eg, Cognitive Processing Therapy) and Mindfulness-Based Therapy (eg, Acceptance and Commitment Therapy) principles and practices. Content is also informed by findings from MST research studies as cited within the “About” section of the app.

JavaScript Object Notation (JSON) formatted usage data were downloaded through a secure data management platform. This platform was developed by the VA National Center for PTSD for quality improvement purposes in accordance with VA’s Technical Reference Model [36]. All data collected were anonymous and, thus, did not include any identifying information about users, such as demographic information. The dataset comprised “event-level” data in which each row (or set of rows) of data represented one in-app event (ie, one app interaction, such as a button press or a swipe). The dataset included the following variables: (1) an anonymous computer-generated alphanumeric identification (ID) string assigned when the app was first downloaded, (2) the operating system (ie, Android or iOS), (3) coded event and metadata information describing the interaction (eg, which button was pressed on which screen), and (4) a timestamp indicating the date and time the event occurred.

ID numbers served as proxies for the identification of individual users. It is possible that multiple ID numbers were associated with a single user if they deleted and redownloaded the app, reset the app, or used the app on multiple devices. Given the inability to differentiate between actual users due to anonymity, and the likely rarity of such occurrences, unique ID numbers represented individual users for the purposes of this study.

Beyond MST usage data from March 11, 2021 through July 29, 2024, were downloaded on July 30, 2024. Data were cleaned, processed, and analyzed using R statistical software (v4.4.1; R Core Team). As part of data cleaning, errors in the raw event-level dataset were identified via descriptive statistics, graphical visualization, and simple univariate and bivariate analyses. Research and testing data were removed. Problematic data (eg, due to technical issues) were removed or corrected. Next, a data dictionary mapping user-app interaction data was constructed to guide summarization of the event-level data into user-level variables with one row per user. User-level variables captured total scores on in-app assessments, duration of app use, frequency of feature access, and frequency of feature use.

Several types of models were constructed to correctly evaluate the impacts of symptom levels on app engagement metrics. Careful attention was given to the data type (ie, count and continuous) and distribution patterns of the outcome variables to inform the optimal model. Rather than include all 3 predictors in a single model, separate models were constructed for each of the 3 predictors on each of the 5 outcomes to maximize sample size and maintain power across the 3 subsamples. Additionally, a quadratic term was included in all models to assess for possible curvilinear relationships.

For 4 count variables (distinct days of use, frequency of tool access, frequency of Learn topic access, and frequency of assessment completion), Poisson and negative binomial models with log-links were evaluated. For distinct days of use and frequency of assessment completions, zero-truncated Poisson and negative binomial models were evaluated as a zero value was not possible. Model fit was compared based on the Akaike Information Criteria statistic, where lower index values indicate better fit [40]. Results from the zero-truncated and standard negative binomial regression models provided the best fit overall and for addressing overdispersion [40]. Thus, those modeling results were used to assess the impacts of predictors on outcome variables. Results are reported as unstandardized coefficients (B). To facilitate interpretation, coefficients were mathematically adjusted as relative percentage changes in expected app engagement using the transformation (e^B – 1) × 100.

For the continuous outcome, minutes of use, robust linear regression was used, which lessens the impact of outliers and nonnormality. Unstandardized regression coefficients (B) were interpreted as the expected change in minutes of use for each 1-point change in the predictor. All 3 predictors were mean-centered to reduce multicollinearity and simplify the interpretation of the regression coefficients.

Figure 2 illustrates the types of individual regression models selected. All analyses were performed using R statistical software (v4.4.1; R Core Team), with a significance level set at P<.05.

The Stanford University Institutional Review Board determined that this project is quality improvement and is not classified as human subjects research and was, therefore, excluded from oversight (protocol 75160).

Subsamples were obtained from a larger sample of users (N=27,517). Since the data were anonymous, sociodemographic information about users was not available. Among a subsample of users who completed at least one PCL-5 (3689/27,517, 13.4%), 1778 (48.2%) were Android users and 1911 (51.8%) were iOS users. Based on the PCL-5, a large majority of users (3293/3689, 89.3%) met criteria for probable PTSD (score of ≥31). In addition, the distribution of PCL-5 scores across users in this subsample was negatively skewed (Figure 3). To account for prorated responses in the PCL-5, a Wilcoxon rank-sum test was performed to assess potential differences in median PCL-5 scores between Android and iOS users. Results indicated a statistically significant difference in scores, W=1,622,530; P=.02, with median scores of 53.7 (IQR 22.1) for Android users and 55.0 (IQR 23.0) for iOS users. However, the effect size was small (r=0.04, indicating that 0.16% of variance in PCL-5 scores can be attributed to differences in Android vs iOS users), suggesting the observed difference (1.3 points) was statistically and clinically negligible. Consequently, data from different operating systems were combined for the main analyses. Descriptive statistics for all subsamples and engagement measures are shown in Table 1.

Users who completed at least one PMBS (2197/27,517, 8%) included 1030 (46.9%) Android and 1167 (53.1%) iOS users. As with PCL-5 scores, the distribution of PMBS scores across users in this subsample was also negatively skewed (Figure 3).

Finally, the subsample of users who competed at least one WEMWBS (2160/27,517, 7.8%) included 1043 (48.3%) Android and 1117 (51.7%) iOS users. In contrast to the distributions for PCL-5 and PMBS scores, the distribution for WEMWBS scores was positively skewed (Figure 3).

Mobile app engagement is a challenging construct to analyze, particularly when examining factors that influence human behavior in naturalistic settings. Researchers have made significant strides in understanding how aspects related to the apps themselves (eg, usability and acceptability) impact mental health app engagement. However, less is known about which user-level factors impact engagement. Emerging research highlights the potential influence of factors such as age, gender, and digital literacy [32]. However, few studies have evaluated how symptom severity might impact engagement with digital health technologies, especially mental health apps for trauma and PTSD. Studies examining other mental health conditions (eg, depression) suggest that symptom severity may play a crucial role [30-33]. This exploratory study furthers our understanding of how posttraumatic symptoms, negative cognitions, and levels of mental well-being may be associated with engagement with an app designed for those who experienced MST.

Based on usage data, more than 27,000 users engaged with Beyond MST from its launch on March 11, 2021, through July 29, 2024. A small subsample of users completed the in-app PCL-5 (3689/27,517, 13.4%), PMBS (2197/27,517, 7.8%), or WEMWBS (2160/27,517, 7.9%) at least once during the observation period. These assessment completion rates are consistent with data from studies on other publicly available mental health apps [33,41-43]. In addition, the subsamples were nearly split in terms of Android (46.9%-48.3%) versus iOS (51.5%-53.1%) users.

The results suggest that Beyond MST is reaching the targeted populations, including those with high levels of PTSD-related concerns, including negative posttraumatic cognitions and poorer mental well-being. Specifically, summary statistics for the subsamples revealed that scores for the PCL-5 and PMBS were positively skewed, indicating relatively severe PTSD symptoms and NPCs, whereas scores for the WEMWBS were negatively skewed, indicating relatively poor mental well-being overall. In addition, almost 90% (3293/3689) of users who completed the PCL-5 met criteria for probable PTSD. Taken together, it is likely that these subsamples of users are highly symptomatic, with substantial posttraumatic symptoms that may negatively impact their well-being and quality of life, similar to findings from studies of the PTSD Coach app [41,42].

Consistent with research on engagement with mental health apps, generally there was limited engagement with the Beyond MST app in terms of duration of app use as well as frequency of feature access and use. The median distinct days of use was just 3-4 days (IQR 5-6), and the median minutes of use was about 22-30 minutes (IQR 33.7-42.9) across samples. The median frequency of coping tool access and Learn (ie, psychoeducation) topic access was 3-5 (IQR 7-9) and 1-2 (IQR 6-7) times, respectively. Finally, the median frequency of assessment completion was 2-3 (IQR 2). Considering the tools are designed to be used multiple times to support coping skills practice and the assessments are meant to be completed repeatedly for progress tracking, it follows that median rates of tool access and assessment completion would be higher compared to psychoeducation access. Both the relatively low rates of engagement and the differences in tool versus psychoeducation access are consistent with findings from evaluations of other mental health apps with similar designs and content areas [33,41-43].

Results also uncovered curvilinear patterns related to trauma-related measures and app engagement. The nonlinear relationships were most clearly seen in relation to PTSD symptom severity and NPCs. For users with the most and least severe symptom levels (eg, PCL-5 score ≥65 or <25), engagement with Beyond MST was lower compared to users reporting moderate symptom levels (eg, scores of 30-40). These effects were consistent for 2 engagement outcomes, distinct days of use and frequency of tool access. This curvilinear pattern is consistent with the Yerkes-Dodson principle [44,45], describing the inverted-U shaped relationship between arousal and performance, with optimal levels of performance, or engagement in this case, at moderate levels of arousal and with decrements when arousal is too low or too high.

For minutes of app use and frequency of assessment completion, the relationship between symptom severity and app engagement was less consistent and clear. For minutes of app use, the results were mixed. Like with PTSD symptom severity and distinct days of use, there was a similar curvilinear relationship between PTSD symptom severity and minutes of use. However, no statistically significant relationship was found for NPC severity and minutes of use. It is unclear why consistent effects were not observed across similar duration of use metrics. One potential factor is the difference in the average minutes of app use between the subsamples (PCL-5 sample: mean 46.6, SD 147.6 minutes; PMBS sample: mean 62.4, SD 188.9 minutes) and increased variability. In contrast, consistent effects were found for distinct days of use where the subsamples had more similar means (PCL-5 sample: mean 6.4, SD 14.2 days; PMBS sample: mean 7.7, SD 17.6 days) with less variability.

These same patterns were observed for frequency of assessment completion where the highest and lowest levels of PTSD symptom severity predicted lower completion rates for all scales compared to those with moderate PTSD symptom levels, whereas NPC severity showed no significant linear or quadratic associations. Assessment completion reflects a complex, multistep behavioral sequence, requiring initiation, sustained attention, and follow-through. Therefore, it is possible that PTSD symptom severity may more strongly disrupt the cognitive and motivational processes needed to complete structured tasks, while NPCs alone may not exert the same level of interference. Other factors, such as the length of the scale (15 vs 20 items) and the trauma-related symptom content of the PCL-5 items may influence engagement.

The pattern of results for psychoeducation access was also inconsistent. There were no curvilinear relationships for either symptom severity measure. In addition, the results regarding linear effects were mixed. Increasing PTSD symptom severity predicted less psychoeducation access, but there was no relationship between NPC severity and psychoeducation access. Although the reasons for this discrepancy are unclear, it is possible that those experiencing particular PTSD symptoms at higher levels are seeking in-the-moment symptom relief versus wanting to learn about MST.

In contrast, greater mental well-being was associated with increased app engagement, generally following linear trends. These patterns were strongest for distinct days of use and tool access, while effects for psychoeducation access were more modest or inconsistent. Notably, those with the lowest levels of well-being accessed the tools less frequently compared to users with higher well-being. Taken together, this suggests that in this sample of primarily highly symptomatic users, individuals displaying better well-being–related factors (eg, positive emotions, life satisfaction, and functioning) may continue to find value in the app over time, using it for ongoing support; however, users with fewer internal resources may find slightly more difficulty sustaining engagement.

Overall, the novel findings of possible inverted-U patterns prompt a need for further assessment of curvilinear relationships in digital mental health engagement research. Potentially, users with lower posttraumatic symptom severity may not perceive a pressing need for support, while those with more severe symptoms may experience avoidance, emotional numbing, or cognitive overload that impairs their ability to engage meaningfully with digital tools. It also follows that emerging evidence of such relationships has been found for depression symptoms (eg, [33]) since depression often co-occurs with MST [5]. This also parallels psychotherapy research showing that both low perceived need and high avoidance can undermine treatment engagement and retention [46-48]. Therefore, it is possible that other symptoms not assessed are driving these relationships. Taken together, engagement is not simply a function of need, but also of readiness and psychological bandwidth.

However, it is important to interpret these findings in the context of clinical significance. According to the clinical guidelines for the PCL-5 [49], a 10-point change in the PCL-5 suggests a clinically significant change in PTSD symptoms. This can be used as a benchmark for understanding clinically meaningful differences in app engagement. For example, based on the model in this study, a 10-point increase on PCL-5 predicts approximately a 10% reduction in the expected number of distinct days of use, which translates to roughly half a day less engagement from a baseline median of 5 days. Similarly, for tools, psychoeducation, and assessments, a 10-point increase corresponds to modest decreases in the expected frequency of access or use. While these effects are statistically significant, the absolute changes in engagement are small, suggesting that even clinically meaningful differences in PTSD symptoms may only slightly reduce platform use. These small effect sizes highlight the importance of interpreting behavioral outcomes in real-world contexts where even brief or infrequent engagement may still yield therapeutic benefit. It is also possible that symptom levels indirectly affect engagement through more direct predictors such as motivation level [50]. This is especially pertinent for types of engagement that require sustained attention and repeated behavior, such as in-app symptom tracking over time or tool use for ongoing skills practice. Finally, since the subsamples displayed high levels of symptoms overall, it is important to interpret these findings in the context of highly symptomatic individuals. For example, compared to users with high levels of symptoms who may still benefit from self-guided use, perhaps those with the highest levels of symptoms might display improved engagement when provided additional support (eg, human coaching) when using apps such as Beyond MST.

Given the curvilinear relationships found, future work may also benefit from using analytic methods capable of detecting more complex or heterogeneous patterns of engagement. The current models tested only linear and quadratic associations assuming a homogeneous population. However, the overdispersion and presence of outliers suggest that the sample may include multiple populations, such as “typical users” and “super users,” who engage with the app differently. Future studies could use alternate methods, such as mixture modeling approaches to determine whether subgroups of users follow distinct engagement patterns or whether more nuanced relationships between symptom levels and app engagement exist.

In sum, PTSD symptom severity and, to a somewhat lesser degree, negative posttraumatic cognitions, may be important factors to consider when addressing user engagement with an app for MST. This has important implications for app developers and mental health professionals. Specifically, personalized in-app treatment plans or stepped care pathways based on baseline symptom level, such as increased human support for those with more severe symptoms, may improve app engagement. Providers using apps in care should also consider symptom severity when making recommendations (eg, psychoeducation vs tool use). Additionally, the insights generated from these analyses suggest how symptom severity can be used to understand digital interventions along the continuum of care. This is especially crucial for apps such as Beyond MST that can be systematically integrated with care in large health care systems serving diverse populations with a range of mental health symptoms and trauma types [51]. Although these findings may suggest potential clinical implications for the subsamples examined, limitations of the current study should be considered when interpreting these results more broadly.

Several study limitations are inherent to the nature of the data collected as well as the subsamples and methods used. First, due to the data being anonymous, user characteristics were not accessible, making it impossible to control for potential confounding factors such as sociodemographics. This may have been problematic because some demographic factors (eg, female gender [4] and young age [52]) are disproportionately associated with risk of MST victimization. More generally, studies have found that women are also more likely to adopt and engage with digital mental health interventions compared to men [32]. Therefore, it is possible that the subsamples examined in this study were not demographically diverse, limiting the generalizability of these results to the broader population. Clinical trials should be conducted to explore the relationship between posttraumatic symptom severity and app engagement while controlling for potential confounding factors.

Second, this study focused on a subset of users who engaged with the in-app assessments, which may limit generalizability to the complete user sample. This potentially biases the results, as these subsamples of users may be more engaged when compared to the overall user base. Nevertheless, there remains value in the evaluation. First, the data provide insights into the real-world use patterns and relationships among large user samples that engaged with a free and publicly available app. For-profit companies often do not release such detailed information around mobile app engagement. These results can inform hypothesis generation for future studies and potential methods for increasing engagement. Regardless, further user evaluation and controlled studies are needed to identify possible differences in engagement between the broader sample of users and the subsets examined in this study.

Third, the in-app assessments and measures of app engagement used in this study may not fully capture the scope of posttraumatic symptom severity and its effects on app engagement. Several different symptoms are associated with trauma and PTSD, so the use of PCL-5 scores does not inform how individual posttraumatic symptoms might predict app engagement. For example, avoidant behavior, a hallmark symptom of PTSD, might negatively impact app engagement when compared to other symptoms [5]. Also, this study focused on objective metrics of app engagement. Future user experience studies with Beyond MST should explore subjective measures of app engagement such as self-reported intentions to engage. This would provide a more comprehensive picture of how users intended to engage, how they actually engaged, and possible barriers to engagement.

Relatedly, there are numerous app engagement measures that were not explored due to the constraints of the app’s design and availability of data. For example, apart from feature access and assessment feature use, other types of feature use could not be assessed due to limitations in the backend of Beyond MST. Importantly, researchers have urged the field to move beyond just measuring “clicks and downloads” when examining app engagement [53]. Engagement is a multidimensional construct that not only includes behavioral aspects (eg, duration of use and frequency of use), but also subjective experiential aspects (eg, focused attention, emotional involvement, and usability) [53] which are often best understood through qualitative and mixed methods approaches. The extent to which users meaningfully apply skills learned or practiced via the app in day-to-day life should also be explored in future work. For mental health apps, this means defining what constitutes clinically meaningful app engagement. For example, the frequency of accessing the Deep Breathing tool in Beyond MST does not fully capture the experiences of users who adopted the skill with minimal app use but continued to apply that skill in daily life. In other words, clinically meaningful app engagement may vary from person to person. Longitudinal studies are also needed to understand how symptom severity may impact engagement over time. Overall, further research is needed that captures the breadth of these complex constructs, providing a more nuanced understanding of how posttraumatic symptoms might impact mental health app engagement.

The Beyond MST app has the potential to support a diverse range of service members and veterans who experienced military sexual trauma, whether they are using the app for self-care purposes, or are using it in conjunction with professional care. Even highly symptomatic users were found to engage with Beyond MST. However, some small effects on user engagement based on posttraumatic symptom severity levels were found which may limit the potential effectiveness of Beyond MST in helping some users cope with trauma, find hope, and enhance their well-being, particularly users with the highest levels of symptoms. Additional research on possible user-level predictors of app engagement, and how specific factors might promote or impede engagement, is needed. Findings from this study suggest statistically significant curvilinear and linear relationships between posttraumatic symptom severity and app engagement, but the clinical implications of these results remain unclear without additional study. Careful examination of the potential predictive power of individual posttraumatic symptoms on a wider range of app engagement measures is needed. Nevertheless, this study may serve as a model for future research in this area and highlights the importance of understanding symptom severity as it relates to user engagement to optimize digital solutions for mental health conditions. Continued exploration in this area will be crucial for developing more effective mental health apps for people who have experienced MST and other types of traumatic stress.