We employed a longitudinal single-group open-trial design. This study was approved by the local institutional research ethics board (2019-157) and registered on Clinicaltrials.gov (NCT04127032). This article presents the initial study outcomes for the first 83 clients who enrolled in the PSP Wellbeing Course (one third of the total expected sample of 250). We have followed the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) statement in preparing the article [26].

This study was performed in the Canadian province of Saskatchewan. There was no specialized ICBT for PSP in Saskatchewan prior to this study. ICBT has been available to Saskatchewan residents through the Online Therapy Unit [27] since 2010, but the numbers of PSP using this service have not been systematically tracked.

All clients who enrolled in the PSP Wellbeing Course between December 5, 2019 and July 2, 2020 consented to our use of their data for research and were included in this study. Enrollment in the PSP Wellbeing Course required clients to (a) be 18 years or older, (b) currently or formerly employed as a PSP (as defined above), (c) have access to a computer and internet service, (d) provide an emergency medical contact, and (e) reside in the province of Saskatchewan (although we later began offering the PSP Wellbeing Course in English and French to residents of Quebec as well). Prospective clients were excluded and referred to appropriate services if they (a) reported a high suicide risk; (b) had attempted suicide or had been hospitalized for a high suicide risk within the past year; (c) reported a primary problem with psychosis, mania, or alcohol and drugs; (d) reported receiving psychological treatment more than twice per month; or (e) reported concerns about ICBT and requested referral to other services in the community.

The PSP Wellbeing Course was adapted from a transdiagnostic ICBT program called the Wellbeing Course, which was developed for use in the general population by eCentre Clinic at Macquarie University, Australia [28-32]. The PSP Wellbeing Course is designed to treat symptoms of depression, anxiety, and PTSD. The course consists of five sequential lessons, which are presented in a slideshow format and focus on (1) introducing the cognitive behavioral model and symptom identification, (2) monitoring and challenging maladaptive thoughts, (3) managing physical symptoms, (4) graded exposure, and (5) relapse prevention. Lesson content is relayed through instructive text, diagrams, and illustrative case stories about PSP. Clients are also given access to downloadable materials, weekly homework assignments, and supplementary resources related to issues not addressed in the lessons (eg, assertiveness, communication, sleep problems). Therapists hold graduate degrees in psychology or social work and provide support to clients by phone or secure email up to twice per week, depending on each client’s preference. Each week, clients complete symptom questionnaires and a reflection questionnaire including both open-ended and closed-ended questions designed to gauge response to the treatment program (eg, what clients worked on; any challenges they encountered; and ratings of helpfulness, understanding, and difficulty). The course is designed to be completed in 8 weeks, but clients are provided with therapist support for up to 16 weeks based on symptoms and preference, and are given access to the course materials for 1 year. We tailored the PSP Wellbeing Course using extensive feedback from PSP stakeholders gleaned through 112 interviews [15] and 132 survey responses [23] (eg, replacing generic case stories and examples used to illustrate treatment concepts with case stories and examples about PSP).

Prospective clients completed a two-stage eligibility screening process consisting of an online screening questionnaire (demographics, clinical history, and the measures described below) and a screening interview by phone. Clients then began the PSP Wellbeing Course. Clients were automatically administered brief weekly questionnaires and a longer battery of questionnaires at 8 weeks, as detailed below. Questionnaires were embedded in the course. Members of our research team followed up with clients by phone or email to encourage them to complete posttreatment measures.

Following intention-to-treat principles, we first created multiple imputations for missing data using predictive mean matching. Imputations for missing data were modeled controlling for lesson completion rates, pretreatment symptom measures, and, if available, weekly symptom measures. We used generalized estimating equations (GEEs) to model the change in symptoms from pretreatment to posttreatment [52]. A gamma distribution with a log-link function was used to model changes between pretreatment and posttreatment as proportional and to accommodate the skewed response distribution [53]. We ran the imputations and GEE models in R version 4.0.3 using the mice [54] and geepack [55] packages.

The effectiveness of the PSP Wellbeing Course was evaluated by examining Hedges g and proportional reductions in symptoms. Hedges g is calculated as the difference in means between pretreatment and posttreatment divided by the pooled SD and multiplied by a small-sample bias correction. It is interpreted similarly to Cohen d but was considered to be a more appropriate measure of effect size for this study given the small size of our clinical subsamples. Some researchers have recommended calculating Hedges g using the SD of the control or pretreatment group instead of the pooled SD when pretreatment and posttreatment SDs are unequal [56]. Indeed, we noted that in the subsets of clients scoring above clinical cut-offs at pretreatment, the pretreatment SDs were much lower than the posttreatment SDs. However, because these subsets represented a restricted range of pretreatment scores, a greater SD at posttreatment would be expected—even in the absence of a treatment effect—due to regression toward the mean [57]. Therefore, we used pooled SDs in calculating Hedges g in the clinical subsets to avoid overestimating effect sizes.

We also examined clinically significant deterioration and recovery for the PHQ-9, GAD-7, and PCL-5. Following the UK National Health Service guidelines, deterioration was defined as an increase of at least 6 points on the PHQ-9 [58]. Following precedents set in previous research using the same outcome measures, deterioration was defined as an increase of 5 points on the GAD-7 [59,60] or 10 points on the PCL-5 [61]. Recovery was defined as a decrease of at least 6 points on the PHQ-9, 5 points on the GAD-7, or 10 points on the PCL-5 that resulted in the patient moving from the clinical range to the nonclinical cut-off scores described in the Measures section above. We examined the uptake, treatment satisfaction and acceptability, and program usage patterns using descriptive statistics in SPSS version 23.

There were 83 clients included in this study. The flow of clients through the study process is shown in Figure 1. Client characteristics are summarized in Table 1.

The effect sizes for each scale, along with the percentage changes and reliable change rates for primary symptom measures are detailed in Table 2. The results indicated large effect sizes on the PHQ-9 and GAD-7, and a moderate effect size on the PCL-5. When examining effect sizes for each primary measure only among clients reporting clinically significant symptoms on that measure at screening, we found large effect sizes on all three measures. In general, we found moderate effect sizes on secondary symptom measures in the entire sample and moderate to large effect sizes when examining only clients reporting clinically significant symptoms at screening; however, the results indicated small effect sizes on the combined SIAS-6 and SPS-6 among all clients and for those with clinically significant SIAS-6 and SPS-6 scores at screening.

Of the 62 clients who completed the WAI-SR and questions related to treatment satisfaction, 54/62 (86%) indicated that the PSP Wellbeing Course had increased or greatly increased their confidence in their ability to manage their symptoms, and 61/62 (98%) indicated that the PSP Wellbeing Course was worth their time. Mean scores on the bond, goal, and task subscales of the WAI-SR were 16.11 (SD 4.64), 14.50 (SD 4.72), and 14.63 (SD 4.07), respectively. To our knowledge, there are no formal guidelines for interpreting WAI-SR scores, but these participant scores fall in the “high/positive” range according to the interpretation of one group of ICBT researchers [62].

All 83 clients included in the analyses accessed Lesson 1 of the PSP Wellbeing Course. Most clients also accessed Lesson 2 (79/83, 95%), Lesson 3 (71/83, 86%), Lesson 4 (64/83, 77%), and Lesson 5 (64/83, 77%). Of the 60 clients who completed 8-week measures, at the time they did so, all had accessed Lesson 1 and Lesson 2, and most had accessed Lesson 3 (58/60, 97%), Lesson 4 (54/60, 90%), and Lesson 5 (45/60, 75%). Across the entire sample, most clients who accessed each lesson also accessed the accompanying DIY guides (92% or more for each lesson), FAQs (84% or more), and stories about PSP (84% or more). Many clients accessed additional resources; the most frequently accessed resources were on relationships with significant others (46/83, 55%), anger (42/83, 51%), communication skills (40/83, 48%), and problem solving and worry time (40/83, 48%). Most clients opted to receive therapist support once per week (74/83, 89%), but some opted to receive therapist support twice per week (6/83, 7%) or only on an as-needed basis (3/83, 4%). On average, clients sent 4.98 messages (SD 5.53) to their therapists and received 9.80 (SD 4.71) messages.

Some clients reported using the PSP Wellbeing Course concurrently with other treatments. At prescreen, some clients reported currently taking one or more medications for their mental health (18/83, 22%) or receiving mental health care from one or more providers (19/83, 23%), the most common of which were family doctors (13/83, 16%), psychologists (6/83, 7%), psychiatrists (4/83, 5%), and counsellors (3/83, 4%). Per the eligibility criteria described above, none of these clients were receiving mental health treatments from these providers more than twice per month. Several clients (4/83, 5%) indicated that they were on a waitlist to see a mental health care provider.

Canadian PSP experience high rates of mental health problems and face several barriers to mental health care [15,16]. ICBT is a promising solution for effectively managing symptoms of common mental disorders [19,20] in a private and convenient fashion that allows clients to overcome key barriers for care [15,17]. PSPNET has developed a tailored ICBT program for PSP called the PSP Wellbeing Course. This article presents the initial program outcomes.

The results indicated that the PSP Wellbeing Course is effective for treating symptoms of depression, anxiety, PTSD, panic disorder, and anger, and also improves clients’ functioning across three domains of life. The course appeared to be only slightly effective for treating symptoms of social anxiety, but this should be interpreted with caution because the subsample reporting clinically significant social anxiety was small (n=27). Nonetheless, in keeping with the learning health system approach [25], PSPNET will explore the possibility of providing additional resources to help clients manage social anxiety symptoms. This is important because approximately 15% of Canadian PSP—and 33% in this study’s treatment-seeking sample—struggle with clinically significant social anxiety [11]. Results concerning treatment satisfaction, working alliance, and engagement were promising. Although in our interviews with PSP stakeholders [15], most had expressed a preference for a flexible treatment duration, it was encouraging that most clients completed all lessons within 8 weeks. It was also very encouraging that working alliance was high because many PSP we interviewed previously [15] reported negative attitudes toward mental health care providers (eg, distrust, discomfort, beliefs that service providers do not understand PSP). Most clients accessed multiple additional resources, suggesting that they found them helpful and lending some support to recommendations for modularized, optional treatment elements in internet interventions [21].

Descriptively, these outcomes are comparable to the most recent published outcomes of the Wellbeing Course offered by the Online Therapy Unit to the general population of Saskatchewan [63]. Effect sizes, percentage changes, and reliable change rates were comparable for all available symptom measures with the possible exception of the SIAS-6/SPS-6, which appeared to show better outcomes for the Wellbeing Course. However, mean pretreatment social anxiety symptoms were considerably less severe in the present sample than in the Online Therapy Unit’s clients [63], and the smaller change in symptoms found in this study may be due, in part, to a floor effect. Of note, the present findings show that the PSP Wellbeing Course was moderately effective for treating anger, which has not been measured in research on the Wellbeing Course. PSP participating in this study reported concurrent use of psychotropic medication less frequently than clients of the Wellbeing Course [63]. Results concerning treatment satisfaction, working alliance, and engagement were similar across the two courses.

This study has several limitations. First, the data were collected during the COVID-19 pandemic, and the impact of the pandemic on outcomes is unclear. Second, there was no control condition and therefore no direct evidence that symptom change can be attributed to the PSP Wellbeing Course. However, it has been argued that the literature on digital mental health interventions would benefit from more research evaluating such interventions in real-world settings [64,65], as we did in this study. Third, at 8 weeks, we were missing PHQ-9 and GAD-7 data for 23% of clients and PCL-5 data for 25% of clients. A conservative approach was used to manage missing data, but it is possible that clients for whom data were missing may represent a unique subset of clients (eg, clients who experienced less positive outcomes).

Further research can extend these findings in several ways. This article describes the results for only the first third of the sample (83 of 250) specified in our trial protocol; full results will be reported when available, including outcomes at 1-year follow-up and for PSPNET clients in Quebec. Further research will also be required to explore demographic and clinical predictors of treatment response for the PSP Wellbeing Course and to identify new ways to tailor and improve ICBT for PSP. Finally, although this study suggests that the PSP Wellbeing Course benefits the PSP who access it, economic evaluation will be required in the future.

Preliminary results indicate that the PSP Wellbeing Course is a promising and effective method for treating symptoms of depression, anxiety, and PTSD among Saskatchewan PSP. Our results also suggest that the course improved clients’ functioning and reduced their symptoms of panic and anger, but not social anxiety. Clients reported good working alliance and treatment satisfaction, and they demonstrated good engagement. Consistent with the concept of learning health systems, PSPNET will continue applying research results to iteratively improve tailored ICBT services, such as by investigating ways to better treat symptoms of social anxiety among PSP. Our findings may also be helpful for other groups involved in researching, providing, funding, or otherwise supporting mental health care for this unique and underserved population.