The University of Pittsburgh Institutional Review Board approved this study. SMs and veterans between the ages of 18 and 60 years were recruited from other studies (Military Operational Medicine Research Program proposal number PT130572, PI: Reifman/Germain; Military Operational Medicine Research Program log number 11293006, PI: Germain; log number 13154004, PI: Okonkwo) that used postcards, flyers, study websites, social media/Facebook (San Francisco, CA), and public television advertisements for recruiting purposes. Since our study required participants to use their own device, eligible SMs and veterans had to both own a mobile phone with internet access and be fluent in the use of that mobile phone. Other eligibility criteria included the presence of a clinically significant sleep complaint as determined by a baseline score of 10 or higher on the Insomnia Severity Index (ISI) [47] and having consistently experienced sleep disturbances for at least 1 month. Participants who were diagnosed with obstructive sleep apnea or who scored greater than or equal to 4 on the STOP-BANG (snoring, tiredness, observed apnea, blood pressure, body mass index, age, neck size, and gender) questionnaire [48-50] were excluded from the study. Other exclusion criteria included a history of psychotic disorder or bipolar disorder, the presence of symptoms of narcolepsy or any other sleep disorder requiring further evaluation and treatment, the presence of any severe or untreated psychiatric disorders associated with marked impairments in functioning, and any scheduled/imminent military deployment during the course of the study. Finally, pregnant or breastfeeding women were not included in the study.

After obtaining each participant’s verbal consent, a telephone screening was conducted to assess eligibility prior to the initial in-person visit. Screening questions were related to the current use of a mobile phone, past and current psychiatric and physical health, and the presence of any suspected or diagnosed physiological sleep disorders or sleep apnea. Eligible participants were invited for an in-person consent and assessment visit.

After obtaining written informed consent, participants underwent a 2-part diagnostic evaluation that included a diagnostic interview and a series of screening questionnaires. The diagnostic interview focused on assessing the insomnia, presence and severity of trauma history, alcohol/substance use disorders, other psychiatric disorders, and current physical health. A weekly consensus meeting was held to review diagnostic information and establish the participant eligibility for the open-trial phase of the study. Participants also completed a series of self-report screening questionnaires:

Because insomnia is the most prevalent sleep disorder among post–9/11 SMs and veterans [59,60], the ISI [47] was used as the primary sleep outcome metric. The ISI is a 7-item self-administered questionnaire that subjectively assesses the severity of a participant’s insomnia symptoms, including level of satisfaction with sleep, noticeability and extent of daytime impairment, and additional concerns caused by sleep problems. Each item has a scale with a range from 0 to 4, with a total score of 10 or higher reflecting the presence of clinically significant insomnia [47]. Daytime sleepiness was assessed using the ESS [58], an 8-item self-report questionnaire where respondents are asked to rate, on a scale from 0 to 4, their usual chances of dozing off or falling asleep while engaged in 8 different activities. The overall ESS score can range from 0 to 24, where a score of 10 or higher indicates clinically significant somnolence.

Overall sleep quality was assessed using the PSQI [55] and PSQI-A [56,57]. The PSQI is an 18-item self-administered questionnaire that assesses different components of sleep quality with scores ranging from 0 to 21. A score of 5 or higher has been shown to reflect clinically significant sleep complaints. Disruptive nocturnal behaviors were assessed using the PSQI-A, which is a 7-item self-report measure that assesses the severity of 7 disruptive nocturnal behaviors commonly experienced by trauma-exposed individuals [56,57]. PSQI-A scores range from 0 to 21, with a score of 4 or higher indicating clinically significant disruptive nocturnal behaviors.

Given the common comorbidity between sleep disturbances and psychiatric symptoms, participants also completed the PCL-C [53], the Patient Health Questionnaire 9-item (PHQ-9) [61] to measure symptoms of depression, and the Generalized Anxiety Disorder 7-item (GAD-7) [62] to measure symptoms of anxiety. The PCL-C is a 17-item self-report rating scale of PTSD symptom severity, with higher scores reflecting more severe symptomatology. The PHQ-9 item assesses the frequency of 9 symptoms of depression over the preceding 2 weeks. Scores of 5, 10, 15, and 20 represent no to mild, moderate, and moderately severe depression, respectively. Finally, the GAD-7 is a brief self-report measure of symptoms of generalized anxiety.

Consistent with the previous trials [17,46], treatment response was defined as a reduction of 8 or more points on the ISI [63]. Furthermore, remission was defined as meeting the treatment response criteria and achieving a posttreatment ISI score below the clinical threshold of 7. Treatment response was also assessed with more global measures of improvements, using the PSQI (defined as a decrease of at least 3 points from pre- to posttreatment) [21,46] and the Patient- and Clinician-Rated Clinical Global Improvement Scales [64-66].

The second aim of the study was to compare clinical improvements in sleep and psychiatric symptoms using iREST relative to habitual, in-person delivery formats of evidence-based behavioral sleep treatments. To do so, we extracted data from 2 previously published trials [17,46]. The first trial included an 8-week treatment arm (CBTI+IRT) that combined in-person CBTI and imagery rehearsal therapy (IRT) for nightmares [46]. The second trial tested an abbreviated, 4-week CBTI protocol specifically designed for SMs and veterans [17]. In this trial, the intervention was delivered during a 45-minute session in week 1 followed by a booster telephone session in week 3 [67]. Both of these trials enrolled SMs and veterans presenting chronic, service-related sleep complaints and employed the same sleep measures (ie, ISI, PSQI, PSQI-A, and ESS). Symptoms of PTSD were assessed with the PCL-C. Symptoms of depression and anxiety were assessed with the Beck Depression Inventory [68] and the Beck Anxiety Inventory (BAI) [69].

Figure 3 depicts this study’s overall workflow. After providing their written informed consent, participants were issued a 6-digit ID and then asked to complete the series of self-report questionnaires aimed at assessing their military history and any demographic variables, baseline sleep quality, current sleep habits and behaviors, current psychological well-being, and overall perceived physical health. Participants also completed clinician-administered interviews to assess the presence of psychiatric disorders. After the interviews, participants downloaded the iREST app on their personal mobile phone and received instructions on how to use the app. They were instructed to complete the morning and evening sleep diary for the next 7 to 10 days, at which point they would receive their personalized sleep recommendations via the app. They were also instructed to contact their clinicians via the text messaging function or by telephone as needed. This first visit took approximately 90 minutes.

After this initial period of 7 to 10 days with the app, participants received their individualized recommendations via the app, with specific instructions on how to implement recommended behavioral changes. Each week, they completed a short battery of self-report measures to assess overall perceived improvements in sleep, side effects, and symptoms of depression, anxiety, and PTSD to monitor progress. Clinicians reviewed participant sleep information and reported symptoms and improvements via the portal. Adjustments to the initial recommendations were provided on average on a weekly basis. During the intervention phase, participants who reported an exacerbation of symptoms were scheduled for telephone or in-person visit. After the intervention phase, participants who continued to experience significant sleep complaints were offered in-person sleep consults with the clinician and/or referral to a sleep clinic or mental health services.

Descriptive statistics were performed to describe the demographic characteristics of the study participants using frequencies for categorical variables and means and standard deviations for continuously measured demographic variables.

SPSS Statistics software version 24.0 (IBM Corp) was used to assess pre- to postintervention changes in sleep and psychiatric symptom severity. For the first aim, paired t tests were used to test pre- to posttreatment differences on self-reported sleep and psychiatric symptom measures. To better contextualize the magnitude of improvement, Cohen d effect sizes were also computed. In addition, mixed model analyses of variance (ANOVAs) were performed to explore whether the improvement in outcomes differed based on the presence or absence of comorbid disorders.

For the second aim, descriptive statistics were performed to describe the demographic characteristics of the study participants using frequencies for categorical variables and means and standard deviations for continuously measured demographic variables. Using a chi-square test for categorical variables and ANOVAs for continuous variables, each demographic variable was compared with the same variable from the previously published in-person trials [17,46] to determine whether there existed any statistically significant differences between the distribution in this study sample and the previous one. Rates of treatment response and remission across 3 delivery formats were compared using the chi-square test. Finally, a mixed model ANOVA was conducted on the primary sleep outcome (ISI) to explore on whether there were different effects between the groups.

A total of 111 SMs and veterans expressed interest in participating in this pilot feasibility study (Figure 4). Of these, 40 provided written consent and 33 were found eligible to complete the study. Out of the participants who started the intervention, 84% (27/33) completed the posttreatment/follow-up assessment and were included in the follow-up analysis.

Of the 5 participants who did not finish the treatment, 3 were excluded due to poor compliance and 2 withdrew because they were no longer interested in the study. There were no significant differences between those who completed the study and those who did not. Table 1 shows the demographics and baseline scores for the 27 completers.

The pre- and postintervention tests show statistically significant improvement in primary and secondary sleep outcomes. As shown in Table 2, the mean reduction on the ISI was 9.96, t₂₆=9.99, P<.001, which reflects a decrease by at least 1 severity category on this measure. Additionally, there was marked improvement in sleep quality, with a mean reduction on the PSQI of 6.67, t₂₆=8.22, P<.001, and mean reduction on the overall severity of disruptive nocturnal disturbances on the PSQI-A of 2.37, t₂₆=3.55, P=.001. Finally, there was a decrease in daytime sleepiness, with a mean reduction on the ESS of 2.04, t₂₆=2.98, P=.006. Clinically and statistically significant improvements in symptoms of depression, anxiety, and PTSD were also detected (see Table 2 for details on these improvements).

Using the global measures of clinical improvement, 74% (20/27) of participants reported that they were much or very much improved posttreatment, whereas clinicians rated 82% (22/27) of participants much or very much improved posttreatment. Using the criterion of a decrease of at least 3 points on the PSQI, 85% (23/27) of patients showed improvements in global sleep quality and, of those, 19 achieved full remission, defined as a posttreatment PSQI score of less than 5. Using the stringent definition of treatment response of a reduction by 8 points or more on the ISI, 70% (19/27) of participants met the criterion for treatment response and 16 presented full remission of insomnia (ie, ISI score less than 7 posttreatment—59% (16/27) of the full sample and 84% (16/19) of responders).

As shown in Table 4, there were no statistically significant demographic differences between participants in this study (iREST) and those in the traditional trials used as a control. Furthermore, participants for the iREST study and prior traditional studies were drawn from the same geographical area. For all 3 studies, participants had to be able to attend an in-person assessment at the University of Pittsburgh Military Sleep Tactics and Resilience Research Team office.

Mixed model ANOVAs were also performed within-group to explore whether the improvement in outcomes differed based on the presence or absence of comorbid PTSD and mood and anxiety disorders. After controlling for baseline ISI scores, the only significant effect observed was expressed in terms of time (pre- to post-). No significant effect from comorbidity conditions or any significant interactions between time and these conditions was observed. See Table 3 for additional results.

On the ISI, the CBTI+IRT, brief CBTI, and iREST yielded large and clinically significant improvements, with Cohen d effect sizes of d=1.50, d=1.96, and d=1.93, respectively (shown in Figure 5). Additionally, significant improvements in sleep quality, or reductions in PSQI score, were also observed in all 3 groups, with d=1.45 in the CBTI+IRT group, d=1.56 in the brief CBTI group, and d=1.58 in the iREST group. There were also significant improvements in PSQI-A scores for the CBTI+IRT and the iREST groups, with d=0.87 and d=0.71, respectively.

PTSD symptoms were similarly reduced in the CBTI+IRT and iREST groups pre- to posttreatment, with Cohen d effect sizes of d=1.08 and d=1.19, respectively; a lower Cohen d effect size of d=0.20 was reported on the brief CBTI group. Depression symptom severity was also significantly reduced in all groups, with d=0.65 in the CBTI+IRT group and d=0.69 in the brief CBTI group compared with d=1.13 in the iREST group. For symptoms of anxiety measured with the BAI, pre- to posttreatment changes in both the CBTI+IRT and brief CBTI groups were nonsignificant (d=0.08 and d=0.14, respectively), whereas the pre- to posttreatment changes in symptoms of generalized anxiety as measured by the GAD-7 were significantly more pronounced in the iREST group (d=0.89).

Furthermore, a mixed model ANOVA conducted before and after treatment, with time functioning as a within-subject repeated measure on the primary clinical outcome (ISI), showed no significant group × time interaction (F_2,53=0.36, P=.70) and no main effect of group (iREST vs brief CBTI vs CBTI+IRT; F_2,53=1.02, P=.37). Instead, only a main effect of time was detected (F_1,53= 140.5, P<.001). This further suggests that iREST may be noninferior to the in-person brief CBTI and standard CBTI.

The response rate for the previous in-person brief CBTI study was 76.47% with a remission rate of 52.94%, while the rates for the CBTI+IRT trial were 70.59% and 64.70%, respectively. No significant difference exists between the rates in our trial and the previous in-person brief and standard CBTI studies (illustrated in Figure 6). Here the chi-square values are χ²=0.22, P=.90, and χ²=0.49, P=.78, for response and remission, respectively.

The purpose of this pilot study was to evaluate clinical feasibility and potential benefits of a novel JITAI app (iREST) for the delivery of evidence-based behavioral sleep treatments. In this study, clinically significant improvements in insomnia, general sleep quality, and disruptive nocturnal behaviors were detected pre- to posttreatment with iREST. Clinically meaningful improvements in symptoms of PTSD, depression, and anxiety were also detected. These findings suggest that iREST, a novel mobile health (mHealth) platform, has a high potential for augmenting the current inventory of evidence-based recommended behavioral sleep treatments.

To explore the potential noninferiority of iREST, we compared clinical outcomes observed with iREST in this study to improvements we observed in previous randomized clinical trials with standard or abbreviated in-person delivery of CBTI [17,46]. Consistent with previous JITAI studies [45,70-73], we found that the implementation of personalized and time-varying JITAI approaches with iREST yielded noninferior outcomes with regard to insomnia or measures of sleep quality and disruptive nocturnal behaviors. The rates of treatment response and remission were also comparable to previously reported rates in CBTI trials [17,46,74-76]. Finally, this exploratory comparison suggested that the magnitude of improvements detected for psychiatric symptoms of PTSD and depression were noninferior with IREST as previously detected in clinical trials. Although improvements in symptoms of anxiety seem to be superior with iREST relative to the standard in-person treatments, the different measures used across trials warrant caution.

The use of a mobile phone app and clinician Web portal, features that allow for real-time monitoring and delivery of personalized treatment prescriptions (eg, bedtime reminders, wake-up alarms, appropriate bibliotherapies, and additional assessments) matching the needs of the individual, contributed to the promising results of this study. Furthermore, delivering the behavioral sleep intervention digitally (through mHealth/mobile app) reduced or eliminated the costs associated with an in-person visit to a sleep clinic (eg, loss of wages and cost for travel, accommodations, and child care). It also potentially addressed access and scalability barriers; through the personalization and prioritization embedded in a JITAI-based system, clinicians could optimize their service so it would reach a greater number of patients while maintaining the same level of care. Last, digital interventions such as JITAI allow seamless integration of patients’ clinical progress and outcomes into the medical center or clinic’s electronic health record system and each patient’s own personal health record. This integration is important in maintaining the continuity of care, especially since insomnia is highly comorbid with other health and psychological conditions [6,77,78].

Inherent to the pilot nature of the study, a first limitation is the relatively small sample size. Therefore, the effect sizes detected in this sample, albeit moderate to large, may be attenuated in a larger, confirmatory noninferiority clinical trial. A second limitation relates to the exclusion of individuals with severe psychiatric disorders or sleep apnea. The high rate of exclusion in this study and previous clinical trials highlights the fact that these disorders are highly prevalent among SMs and veterans [59,60] and hence limit the generalizability of the findings to the more severely affected populations. In a related manner, the inclusion of military SMs and veterans may limit the generalizability of the findings to the general civilian population. Future studies should include a wider set of participants and narrower exclusion criteria to assess the effectiveness and generalizability of the treatment for patients with comorbidities.

In this preliminary study, iREST, a novel JITAI app, was associated with statistically significant and clinically meaningful improvements in sleep and psychiatric symptoms in a sample of SMs and veterans with chronic, service-related insomnia. Exploratory comparisons strongly suggest that iREST is noninferior to the traditional in-person delivery formats for CBTI for sleep and related psychiatric symptoms. Together, these findings support the notion that iREST and the JITAI approach can be an acceptable and effective approach to enhance the scalability of evidence-based behavioral sleep treatments. Larger confirmatory noninferiority trials are needed in order to fully understand the effectiveness of JITAI-based iREST among military and civilian populations.