A quasi-experimental mixed methods design was used, with a single-arm group being compared with data from a separate RCT [30,31]. The single-arm group received access to the EPIO program with brief information upon download only (ie, EPIO_NoFollowUp). Outcome measures and program use data were collected over 12 months, and findings were then compared with data from a previously conducted RCT [30,31], in which participants had either received EPIO through a simple blended care delivery model (EPIO_BlendedCare) or had been randomized to a care-as-usual control group. Recruitment for the single-arm EPIO_NoFollowUp group took place after RCT recruitment completion (ie, the 3 groups were not enrolled simultaneously).

Participants in the study were people living with chronic pain for 3 months or more. Participants in the EPIO_NoFollowUp group were recruited between January and May 2022, whereas participants in the EPIO_BlendedCare and care-as-usual control groups (ie, RCT [31]) were recruited between November 2019 and February 2021. Inclusion criteria included (1) living with chronic pain for ≥3 months, (2) being aged ≥18 years, (3) having access to a smartphone or tablet, and (4) being able to understand and read Norwegian. In addition, participants in the RCT had to be able to attend a face-to-face introduction session. Exclusion criteria included migraine or cancer-related pain, as well as untreated severe psychological illness (ie, all self-reported).

The EPIO program consists of 9 CBT-based modules with aspects of ACT, centers on established pain self-management features, and each module contains a combination of psychoeducational content and exercises (eg, diaphragmatic breathing, visualization, mindfulness, progressive muscle relaxation, and thought challenging) [16,32]. The modules include the following main topics: (1) about pain, (2) activity pacing, (3) thoughts and emotions, (4) stress and coping, (5) what is important to me, (6) health habits, (7) communication and social support, (8) coping in adversity, and (9) reflections and the road ahead [16,32]. An animated bird (ie, the EPIOS avatar) accompanies users throughout the program. Further details related to functionalities and the design and development of EPIO can be found in existing publications [16,25,26]. Figures 2 and 3 provide screenshot examples from the EPIO program.

Sample size in the RCT (ie, EPIO_BlendedCare and care-as-usual control) [31] was based on studies with comparable samples [57,58] reporting Cohen d effect sizes of 0.30 to 0.40 on the primary outcome, pain interference. Estimating an α of .05% and 80% power, a sample size of 200 participants was required to detect d=0.4. To account for attrition and provide adequate power for secondary analyses, the total RCT study sample included 266 participants [31]. Sample size in the EPIO_NoFollowUp group was calculated based on the anticipated change of 0.6 in the control group and a slightly greater change of 0.75 points in the EPIO_NoFollowUp group from baseline to the 12-month follow-up, with a common SD of 0.3. To show a between-group difference in change of 0.15 as statistically significant, a sample of 64 participants in each group was required. Accounting for at least 10% attrition, a sample size of 74 participants in the EPIO_NoFollowUp group was estimated.

Sociodemographic characteristics and user patterns were summarized with means and SDs for normally distributed variables and medians and IQRs for variables with skewed distributions. Categorical data were presented as counts and percentages. All analyses were conducted according to intention-to-treat principles, including all participants regardless of EPIO intervention use (ie, use in the EPIO_NoFollowUp and EPIO_BlendedCare groups). To explore potential between-group differences in outcome measures, the following analyses were performed:

Analyses comparing program use between EPIO_NoFollowUp and EPIO_BlendedCare were performed using the independent-samples Mann-Whitney U test and chi-square test. There were no missing data at the item level in the completed outcome measures, as participants completed outcome measures digitally and could not proceed without completing all questions. Participants completing all outcome measures at the 3-, 6-, or 12-month assessment were referred to as “responders,” whereas those not completing outcome measures were referred to as “nonresponders.” To compare responders and nonresponders (ie, sensitivity analysis) on baseline sociodemographic characteristics (ie, age, sex, marital status, education, employment, disability benefits, income, and years living with pain), the chi-square test for categorical variables and the Mann-Whitney Wilcoxon test was used for continuous variables. P values <.05 were considered statistically significant. In addition, the Little MCAR (missing completely at random) test [59] was used to compare all outcome measures between responders and nonresponders. Findings from these comparisons were not statistically significant at any assessment point (P<.98); thus, the missingness was considered to be completely at random, and multiple imputations were therefore not needed. As all analyses were considered exploratory and hypothesis generating, findings are presented as unadjusted P values and 95% CIs without multiplicity correction, consistent with methodological and regulatory guidance for exploratory work and focused on point estimates and their precision (eg, 95% CIs rather than P values) [60,61]. Statistical analyses were conducted using SPSS (version 29; IBM Corp) and Stata (version 19; StataCorp LLC).

While quantitative data were collected to explore the potential effect of program access (ie, change from baseline in pain interference, pain severity, depression, anxiety, self-regulatory fatigue, HRQoL, pain catastrophizing, and pain acceptance) as well as actual program use (eg, program progress and frequency of use), qualitative data were collected primarily to explore participants’ perceived experiences with use of the EPIO program, including the lack of follow-up during the study period.

Prior to conducting the qualitative interviews in this study, a study-specific matrix was used to strive for a representative sample of the EPIO_NoFollowUp study population in terms of sex, age, years living with pain, and program progression (ie, low, moderate, and high) at 12 months. A subset of participants completing the 12-month follow-up outcome measurements in the EPIO_NoFollowUp group was subsequently identified and invited to participate in a poststudy semistructured interview to explore aspects of program use and impact. As no new information emerged after 15 interviews, data collection was concluded (ie, saturation). Qualitative data derived from the postintervention interviews were analyzed using a rapid analysis inspired by Gale et al [62]. Coauthor EFN read all transcripts to achieve a holistic impression, and data for each participant were categorized based on themes from the semistructured interview guide, including codes and quotes. All codes were then merged into a Microsoft Excel spreadsheet, sorted into subthemes, and the draft was reviewed and modified based on discussions between coauthors EFN and EB [62]. Illustrative quotes were subsequently compiled. Coauthors EFN, EB, and LSN further discussed the findings until reaching and securing consensus.

The study was approved by the Regional Committee for Medical and Health Research Ethics (REK; 2018/8911) and the Hospital Privacy Protection Committee (Personvernombudet; PVO) Institutional Review Board equivalent (2017/6697), and followed national and institutional privacy regulations and guidelines for data storage and handling. Prior to participant inclusion, the study was registered at ClinicalTrials.gov (NCT03705104). Study methods and results are reported in accordance with the CONSORT (Consolidated Standards of Reporting Trials) checklist (Multimedia Appendix 3) [63].

Signed informed consent forms and contact information were stored separately from study data in line with institutional regulations. Outcome measures were collected digitally through the secure TSD platform, and all personally identifiable information was deidentified before study data were exported to a local secure server for further analysis. Participants received no compensation for study participation.

A total of 74 participants living with chronic pain were enrolled in the EPIO_NoFollowUp group, with 1 participant withdrawing prior to receiving the EPIO program, resulting in n=73 for the group. In the RCT, 266 participants were enrolled, with 7 participants withdrawing or failing to respond prior to the introduction session, resulting in n=259 (intervention group, n=125; control group, n=134) [30,31]. Figure 4 provides the trial recruitment and participant flowchart overview.

Of the participants in the EPIO_NoFollowUp group, 15 were interviewed after completion of the 12-month outcome measures. A total of 25 participants in the EPIO_BlendedCare group were previously interviewed after study completion [32]. The total number of participants in the EPIO_NoFollowUp and RCT (ie, EPIO_BlendedCare and care-as-usual control) groups was N=332, with a median age of 49 (IQR 39-55) years at inclusion, and participants were primarily female (271/332, 82%). Most participants reported being on sick leave or receiving disability benefits (232/332, 70%) at the time of enrollment and reported having lived with pain for 10 years or longer (201/332, 61%). No statistically significant differences in sociodemographic characteristics were observed at baseline, although participants differed somewhat in self-reported pain conditions (ie, participants in the EPIO_NoFollowUp group reported a higher percentage of nonspecific disc disorders and the category “other,” as well as a lower percentage of fibromyalgia, compared with the 2 RCT groups) and years living with pain. Sensitivity analysis showed no statistically significant differences between responders and nonresponders regarding sociodemographic characteristics (ie, age, sex, marital status, education, disability benefits, income, and years living with pain), except for sex at 3 months, when a significantly larger proportion of males (ie, considering that only 12 males were included in total) dropped out of the EPIO_NoFollowUp group, and for marital status at 12 months, when more single or divorced participants dropped out (Multimedia Appendix 4). Table 1 provides baseline sociodemographic and disease-related characteristics.

Results from the comparison of the EPIO_NoFollowUp group and the care-as-usual control group showed significant between-group differences in favor of the EPIO_NoFollowUp group in terms of improved HRQoL subscale bodily pain at 6 months (MD 5.7, 95% CI 0.4-11.1; P=.04; ES 0.15) and general health at 6 months (MD 5.7, 95% CI 0.8-10.5; P=.02; ES 0.16), reduced pain catastrophizing for the helplessness subscale at 6 months (MD –2.1, 95% CI –3.8 to –0.4; P=.02; ES 0.17), and improved chronic pain acceptance for the willingness subscale at 3 (MD 0.8, 95% CI 0.0-1.6; P=.046; ES 0.14) and 12 months (MD 1.4, 95% CI 0.2-2.7; P=.03; ES 0.15) and activity engagement at 3 months (MD 1.0, 95% CI 0.2-1.9; P=.02; ES 0.17), as well as total chronic pain acceptance at 3 months (MD 1.8, 95% CI 0.5-3.1; P=.007; ES 0.19). Certain trends toward between-group differences in favor of the EPIO_NoFollowUp group were also noted. Tables 2 and 3 provide outcome measure group comparisons at baseline and at 3, 6, and 12 months.

Results from the comparison of the EPIO_NoFollowUp group and the EPIO_BlendedCare group revealed only 1 significant between-group difference, with a statistically significant reduction in self-regulatory fatigue (MD –2.9, 95% CI 0.3-5.4; P=.03; ES 0.16) at 3 months in favor of the EPIO_BlendedCare group. Details are provided in Multimedia Appendix 5.

Participants in the EPIO_NoFollowUp group used the EPIO program significantly less than those in the EPIO_BlendedCare group. Table 4 provides details. In the EPIO_NoFollowUp group, the median number of days of program use during the 12-month trial was 10 (IQR 3-24) days, compared with 30 (IQR 13-44) days in the EPIO_BlendedCare group. The same pattern was seen in module completion status, with 23% (17/73) completing ≥7 modules in the EPIO_NoFollowUp group at 12 months, vs 62% (78/125) in the EPIO_BlendedCare group (P<.001). Participants completing all modules accounted for 12% (9/73) in the EPIO_NoFollowUp group vs 45% (56/125) in the EPIO_BlendedCare group (P<.001).

Findings show how the type of delivery model may impact the use of and benefit from digital self-management interventions. In the current study, a statistically significant impact was detected for the EPIO_NoFollowUp delivery model, compared with controls, in terms of improved HRQoL, reduced pain catastrophizing, and improved pain acceptance. Participants receiving the EPIO pain self-management program with no follow-up did, however, use the program significantly less than those receiving the EPIO_BlendedCare model, and the majority of participants receiving EPIO_NoFollowUp indicated that some form of follow-up would have been useful and appreciated.

The study also indicates how “one size does not fit all,” as people with a higher education level, despite having no follow-up, appeared to use the intervention more and benefit more (ie, in terms of improvement in pain severity, anxiety, HRQoL, and pain catastrophizing), compared with those with lower education levels. For participants receiving EPIO through the simple blended care model, formal education level did not seem to make a difference.

In line with the original RCT (ie, EPIO_BlendedCare vs care-as-usual control groups) [31], no significant between-group differences were detected for the primary outcome measure of pain interference with function. The findings did, however, also in line with results from the RCT [31], suggest more significant changes in psychological rather than physiological variables for the EPIO_NoFollowUp group compared with controls, likely at least partially due to the psychosocial self-management concept of the EPIO intervention [16,31].

Even without follow-up (ie, EPIO_NoFollowUp), people living with chronic pain having access to EPIO reported significantly improved HRQoL in terms of less bodily pain (ie, at 3 months) and improved general health (ie, at 6 months), as well as significantly reduced catastrophizing about pain in terms of feelings of helplessness (ie, at 6 months), compared with participants without access to EPIO (ie, care-as-usual controls). The findings were supported by qualitative input describing improved awareness related to their own thoughts, feelings, and how to live with the pain (ie, rather than fight it), as well as having acquired coping strategies that helped them deal with pain and the many pain-related aspects in new and beneficial ways. Findings also showed improved acceptance of pain (ie, improved activity engagement and pain acceptance total at 3 months, and improved willingness for pain acceptance at 12 months) compared with controls, and qualitative analyses supported this notion.

Access to digital pain self-management interventions, even without follow-up, may hence contribute to improved QoL and coping with pain in day-to-day life. Considering the potential decrease in impact from 6 to 12 months (ie, apart from willingness to accept pain), it is possible that actual improvement in activity engagement must take place for willingness to accept pain to occur. These findings indicate that access to a digital pain self-management intervention such as EPIO, even without follow-up, could contribute to pain acceptance and hence better QoL. For example, findings from an RCT administering a brief ACT intervention to public health workers experiencing chronic stress and pain indicated intervention benefits, including having fewer sick days and using less medical treatment, compared with controls [65].

Directly comparing participants in EPIO_NoFollowUp with those receiving EPIO_BlendedCare yielded only 1 significant between-group finding (ie, reduction in self-regulatory fatigue in favor of the EPIO_BlendedCare group at 3 months). With neither delivery model appearing superior to the other, this study supports research indicating that both guided and unguided digital interventions can be effective [66]. Results from the EPIO RCT did, however, reveal significant benefits for people with chronic pain having access to EPIO through a simple blended care model (ie, EPIO_BlendedCare), compared with controls, including reduced anxiety, depression, self-regulatory fatigue, and pain catastrophizing, as well as improved HRQoL for the intervention group [31]. This points to the blended care delivery model, when compared with controls, as superior. The current findings are hence in line with those of a 3-arm RCT examining the effectiveness of guided and unguided digital ACT-based intervention for chronic pain [33], in which the guided version, compared with controls, showed a stronger impact than the unguided version, but with no major differences in terms of effect between the 2 intervention groups (ie, guided vs unguided) [33].

Postintervention qualitative findings (ie, using a sequential mixed methods approach) contributed to an enrichment of the study findings, identifying perceived experiences with the use of EPIO, despite the no-follow-up delivery model, to include changes in awareness, acceptance, and coping. These findings are in accordance with findings from the EPIO_BlendedCare group, where the identified main themes and ensuing subthemes included perceived changes in cognition (ie, subthemes insights and self-awareness, acceptance, and shifting focus), changes in coping (ie, subthemes pain, emotions, and activity pacing), and content functionality-specific engagement (ie, subthemes breathing and other mind-body exercises, thought-reflection exercises, and functionalities) [32].

Actual use and intervention adherence are crucial to induce effect, and when intervention engagement and adherence are low, the intended intervention effects are unlikely. The well-known ≤50% adherence challenges associated with digital health interventions are concerning [67,68], and studies point to the necessity of some form of guidance to enhance intervention adherence and hence impact [33-35].

This notion is supported by the current findings, with participants in the EPIO_NoFollowUp group clearly using the program significantly less (median 10, IQR 3-24 days of use) over the 12-month study period compared with those in the EPIO_BlendedCare group (median 30, IQR 13-44 days of use). Similarly, only 23% (17/73) of participants in the EPIO_NoFollowUp group completed ≥7 modules, compared with 62% (78/125) in the EPIO_BlendedCare group. While postintervention interviews identified EPIO as useful and easy to use, with highly relevant content, which is in line with previous qualitative findings [28,29,32,69] regardless of delivery method, a majority of participants in the EPIO_NoFollowUp group said they would have appreciated some form of contact and follow-up from health care providers or project team members.

Supporting recent research, these findings underline the importance of some form of guidance to foster intervention adherence. For example, a systematic review points to the necessity of reporting on engagement data to strengthen the development and effect of digital pain self-management interventions [70]. Other findings also support this notion. For example, in a meta-analysis examining the impact of guidance in mental health interventions, guidance apparently increased intervention completion, and completion rates were 12% higher in guided vs nonguided interventions [71].

Despite the apparent vital relationship between lack of follow-up and program use, several participants did describe having used the program exercises without using the EPIO app after initial practice, which highlights a form of use not captured through quantitative program use measures. This might also potentially explain the detection of some effects despite low intervention use in the EPIO_NoFollowUp group. Previous research has also shown that simply having access to a program or having a contact option can be helpful and contribute to health benefits [72,73].

Education level may impact benefit from health interventions. Higher education has, for example, been associated with profiting more from CBT-type approaches to self-management [74]. Participants with higher education in the EPIO_NoFollowUp group in the current study reported improvement in terms of pain severity, anxiety, HRQoL (ie, subscales role-physical, vitality, and mental health), and pain catastrophizing during the 12-month study, compared with those with an elementary or high school education. Formal education level has also been associated with intervention adherence and attrition [75], and participants with higher education in the EPIO_NoFollowUp group completed significantly more program modules compared with those with a lower education level.

Education level did not appear to influence outcomes or adherence to the same extent for participants receiving the EPIO_BlendedCare delivery model, although individual days of use were significantly higher among participants with higher education in both intervention groups. These findings suggest that higher education levels might be “protective” and associated with program adherence even without follow-up, while the impact of education level appears less pronounced when the intervention program is combined with even simple blended care, indicating that while people with lower education levels may require more support to achieve an effect, even minimal contact such as the simple EPIO_BlendedCare model could potentially be of benefit.

Lower education levels have been correlated with lower health literacy [39,76], that is, the knowledge and ability to access, understand, appraise, and use health information and services [77], which, in turn, may impact the capacity for self-management [78]. Digital and digital health literacy should therefore be considered when designing and delivering digital health interventions [79], and higher digital health literacy has, in fact, been associated with better self-management, better psychological well-being and QoL, as well as better participation in health-related decisions [80].

This study had some major strengths in being able to compare findings from blended care and no-follow-up delivery models of a digital pain self-management intervention already shown to be effective in an RCT. The study also had limitations. First, the EPIO_NoFollowUp group was not a true part of the RCT, as recruitment for this group did not commence until after all participants had been included in the RCT. That is, only 2 of the 3 groups examined in the current study were randomized simultaneously. Comparing postintervention outcomes across groups from 2 different studies, measured at different time points, could introduce bias and interpretive challenges. There is also the issue that the EPIO_NoFollowUp group was recruited in 2022, whereas the RCT groups were recruited in 2019-2021. Some participants were hence involved in the study during the COVID-19 pandemic, which in itself could have impacted study comparisons. The target group, recruitment methods, and study procedures were, however, identical across studies and time points, with the same project team, study area, personnel, and locations, which could aid in minimizing potential differences in all other controllable study aspects.

Second, the number of participants in the EPIO_NoFollowUp group was smaller (ie, approximately half) compared with the 2 RCT groups, which may have impacted study power (ie, insufficient statistical power) and hence between-intervention comparisons related to outcomes. Potential impact of sample size on program use is, however, unlikely. Third, the number of outcomes, subscales, and time points tested in this study could potentially lead to type I errors. However, as all analyses were considered exploratory, no correction for multiple testing was made. The findings should nevertheless be interpreted with caution and corroborated by future, independent datasets. Fourth, exploratory analyses related to the potential impact of education level included small numbers, as only 51% (37/73) of the EPIO_NoFollowUp group completed the 12-month outcome measures (ie, compared with 99/125, 79% in the EPIO_BlendedCare group). Findings from these analyses should therefore also be interpreted with caution.

Fifth, there were some baseline differences between the EPIO_NoFollowUp group and the RCT groups related to self-reported pain conditions, including unspecified disc disorder, fibromyalgia, other types of pain, and pain duration. Pain condition or duration did, however, not appear to impact the findings in the RCT [31]. Also, the fact that these conditions were self-reported, and that participants could report more than 1 condition if they wished to do so, made identifying the potential impact of these reported differences challenging. Sixth, the poststudy interviews were conducted with participants completing the 12-month outcome measures, which may have contributed to an overrepresentation of people particularly interested in the program and program completion. The primary goal with the postintervention interviews was, however, to explore perceived experiences with the use of EPIO, including the lack of follow-up during the study, and the final sample interviewed entailed a carefully selected variation in degree of program progress and/or completion. Also, including people who had not used the EPIO program much could potentially have introduced recall bias. Finally, the majority of participants in the study were, as in most self-management interventions, female, which presents limitations for generalizability.

Digital self-management interventions can be valuable support for people living with chronic pain, their health care providers, and society itself [15]. While delivery models with human support have been the most prevalent in digital health interventions so far [81], in-person guided or blended care delivery models require resources, and with health care systems under pressure [36], effective yet resource- and cost-effective interventions are clearly needed [15]. Unguided digital health interventions for chronic pain have also been shown to require lower costs, potentially being cost-effective additions to more conventional pain interventions [82].

This study implies that even user-centered, evidence-based pain self-management interventions may depend on some degree of follow-up to induce an effect. In real-world settings, mobile health apps are typically downloaded individually (eg, through the Apple App Store or Google Play Store), without additional support or follow-up. Findings also suggest that education level might impact benefit from digital interventions, particularly when delivered without follow-up. Future research should further explore these aspects, as tailoring might be essential when providing access to digital health or pain management interventions [14]. Given clinical resource limitations, providing blended care approaches to those potentially needing them most to achieve the intended effect should be explored. Some type of tailored support may also be cost-effective, and future research should ideally establish the cost-effectiveness of various levels of blended care, stratified by baseline characteristics of target users (eg, education, health literacy, and digital literacy).

Health care services under pressure further highlight the need to improve uptake, engagement, and impact of evidence-based, user-centered digital self-management interventions, without the need for pervasive health care provider involvement [83]. Taking these issues into account, factors such as access to user guidance and health information, information about program progress and feedback features, reminders and self-monitoring aspects, motivational messages, goal setting, social interaction, gamification and rewards, as well as low program cost, should further be considered essential in the process of facilitating uptake, adherence, and engagement with digital health interventions [81,84]. Findings from the current study are in line with these recommendations, as participants, for example, described having wished to participate in the study because the program was evidence-based, which created trust and interest.

Digital pain self-management interventions can be of benefit to people living with chronic pain and provide important means of support for health care providers and society at large. Findings from the current study do, however, suggest that “one size does not fit all,” and when people are to engage in self-management without any form of follow-up, they might not use the intervention as much and subsequently may not benefit as intended. Methods of delivery hence emerge as essential when aiming to improve not only outreach and availability but also, essentially, use and effect. Guided interventions appear to have the greatest impact. However, education level could potentially play a role, particularly when digital interventions are delivered without guidance or follow-up. When clinical or health economy resources are limited, intervention tailoring might be necessary, with delivery with limited follow-up for some, yet blended care models with some form of guidance delivered to others.