The implementation of digital health services in specialized health care services necessitates an understanding of the purpose and use of digital services by both patients and health care workers [1,2]. The potential of digital health solutions is recognized by health authorities and stakeholders; however, despite systematic reviews exploring digital health services, the impact of digital solutions on resource use and patient outcomes, such as symptoms, self-management, quality of life, digital health literacy, and satisfaction, remains uncertain [2-9]. Digital services may improve resource use and provide patients with supplementary and understandable health information; the potential benefit and relevance of digital health solutions increase as medical advancements extend life expectancy and as patients with chronic or long-term conditions in need of frequent and repeated care use a large proportion of the available consultations [9,10]. Exploring how to exploit this potential is crucial, as the increase in patients, combined with constrained health resources, presents obstacles to the sustainability of current health services [11]. Furthermore, reductions in hospital beds may increase the need for more outpatient consultations, which is an arena where digital health services have the potential to aid timely and appropriate care.

To fulfill the potential of digital health solutions, attaining a certain level of health literacy is pivotal because health literacy positively influences both self-management and health-related decisions [2,12-15]. Health literacy is often defined, according to Nutbeam [16], as “the cognitive and social skills that determine the motivation and ability of individuals to gain access to, understand, and use information in ways which promote and maintain good health.” Consequently, digital health literacy concerns the use of digital solutions for promoting and maintaining good health [2]. Low health literacy is associated with poor health outcomes and less digital solution use [4,9,14,17,18], while improved health literacy may enhance self-management and the benefits of digital health solutions [1]. Thus, in accordance with a conceptual model proposing how health literacy is affecting interaction with services and health care workers and outcomes [15], we argue that digital health interventions specifically allowing for more patient-centered and flexible care might support patients’ health literacy and aid in their self-management [19]. However, there is a lack of research exploring the role of health literacy in digital health interventions, although the capacity to benefit from digital health solutions and improve self-management is likely to be enhanced by improved health literacy [2,9]. In addition, there exists a similar gap in knowledge on the relation between digital health literacy and digital health interventions [9].

Digital health solutions in outpatient care typically enable the subjective reporting of health parameters through patient-reported outcome (PRO) measures. PRO measures may aid in self-management and communication between patients and health care workers [20]. Digital engagement, self-monitoring, and data sharing offer advantages for patients; however, whether a digital health solution is meaningful depends on its usability, clinical relevance, and convenience, along with whether it has an evidence-based design [9,20-22]. Recent studies indicate that digital solutions in outpatient care can prevent complications, foster patient engagement, and boost confidence and autonomy [23-26]. However, research on multicomponent digital solutions encompassing PRO measures, asynchronous messaging, remote monitoring, patient notifications, and video consultations is limited. Positive effects have been noted in cancer treatment [24], musculoskeletal pain disorders management [25], and home monitoring for interstitial lung disease (ILD) [23]. Mobile apps for patients with epilepsy have shown promising results; however, the value of patients’ collaborating with health care workers remains unclear [26]. Challenges persist, such as integrating electronic health records and standardizing data for registry use. In summary, studies exploring the implementation and impacts of digital health solutions in outpatient care are warranted.

The objective of this study was to evaluate whether digital outpatient care for 6 months resulted in improved health literacy, health-related quality of life (HRQoL), digital/eHealth literacy, and the use of health care services compared with usual care.

Specifically, we hypothesized that digital outpatient care would be associated with a favorable increase in patients’ ability to understand health information well enough to know what to do. By enabling digital access to services, facilitating patient reports and self-monitoring, and promoting interaction with both the digital solution and health care workers, our intervention reinforces a conceptual model for associations between health literacy and health outcomes [15,19].

We conducted a prospective, multicenter, nonrandomized controlled trial with 2 arms and a 6-month follow-up, using a longitudinal design with 3 assessment points: baseline, 3 months, and 6 months (from September 2021 to June 2023). The control arm received the usual care, while the intervention arm received the digital outpatient care intervention. The trial was registered in Clinical Trials and the International Registered Report Identifier (DERR1-10.2196/46649). The evaluation was based on elements of the method for assessment of telemedicine [27], and our study was reported according to Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist [28], while the PRO measures were reported according to the Standard Protocol Items: Recommendations for Interventional Trial (SPIRIT) PRO Extension [29]. The development of the intervention and study protocol have been reported in detail elsewhere [19].

Patients were recruited from the department of respiratory diseases, the department of neurology, and the department of pain management and research at the Oslo University Hospital as well as the department of cancer at the University Hospital of North Norway. Eligible patients were required to be aged ≥18 years, based at home, and capable of completing questionnaires in Norwegian. Inclusion criteria were applied to patients who were newly diagnosed or referred or to those with a diagnostic history in outpatient care. Patients with cancer had to receive active treatment, have a life expectancy of at least 6 months, and have an expected need for services beyond 6 months. Patients with ILD were eligible unless they had a very low degree of function with cognitive impairment. Patients with epilepsy were included unless they had complex causes, severe comorbidities, or ongoing assessments or treatments. Patients with long-term pain were eligible if they underwent drug testing and self-administered their medication and were not eligible if they had comorbidities that directly affected drug adjustment, had cognitive impairment, or did not live at home.

Eligible patients were identified through consultations or patient lists provided by health care workers from each department. The patients received project information before written consent was obtained. At the 3- and 6-month follow-ups, the patients received either an SMS text message or an email invitation to complete their follow-up questionnaire. Consent and all questionnaire responses were provided through the encrypted version of the web-based questionnaire service Nettskjema (University of Oslo) [30]. The data were stored using a service for sensitive data (TSD), complying with the Norwegian Personal Data Act and Health Research Act. This service required a governmental ID portal for log-in, hence ensuring secure data harvesting. In the event of no response after 1 week, a reminder was sent, followed by a phone call 1 week later for pending questionnaires.

Consent and baseline questionnaire completion were prerequisites for a fulfilled allocation for both the control and intervention participants. In addition, this completion of consent and baseline questionnaire was required to access the digital intervention in the intervention arm to reduce contamination in the baseline questionnaire responses.

Participants recruited in the control arm received usual care per routine at the outpatient clinic where they received care. This included the possibility of contacting outpatient departments by telephone during office hours and regular in-person consultations.

The details of the development and tailoring of the intervention have been previously presented [19]. The Dignio Connected Care platform facilitated timely, personalized contact between patients and health care workers [31]. This platform, including the Dignio Prevent system for health care workers and the MyDignio patient app, is Conformité Européene marked, compliant with privacy regulations, and successfully applied in various global clinical settings. Customizable to individual patient needs, components were added for personalized care, with interface examples provided in Multimedia Appendix 1.

Health care workers from each department assigned tasks to patients in Dignio Prevent and set thresholds for individual patients using a traffic light model. Notifications alerted health care workers if measures deviated, such as patients reporting an increase in pain and side effects, or if the task went undone. The patients received reminders through the MyDignio app and an SMS text message if the task remained undone. Two-way messaging enabled asynchronous communication, allowing patients to ask questions or provide relevant information to their health care workers; likewise, health care workers could respond in due time based on the urgency of the message and their workload. For instance, treatment plans were shared with the patients through the chat messaging function, allowing them to always have access to their current plan. MyDignio app’s information page, which was updatable to health care workers, allowed for individualized updates and templates for multiple patients if needed.

The intervention’s core focus across all 4 departments was improving outpatient service accessibility. The app allowed patients to engage with PRO measures, self-monitor relevant parameters, and communicate asynchronously with health care workers (Table 1). Designated staff assessed the responses, allowing for flexible patient care, which is a strategy aligned with a conceptual model linking health literacy to service access, use, interactions between patient and health care worker, and self-management [15,19].

Patients in the intervention arm were briefly introduced to the app, but no extensive training was provided because the MyDignio app was set to be intuitive. Those using spirometers were instructed to ensure the correct technique and synchronization with the MyDignio app. The intervention arm participants could contact health care workers or researchers for technical assistance.

Most health care workers were involved in developing the intervention; therefore, they were familiar with the platform. The training on interpreting patient-reported scores was department specific. The designated health care workers received compensation for 10% to 20% of their time.

The a priori sample size calculation was based on the HLQ domain 9, “understanding health information well enough to know what to do,” and provided a change from baseline to 6-month follow-up, with an effect size of 0.5-unit difference between the arms on a scale of 1 to 5, with a 20% dropout, a power of 0.90, an SD of 0.6 from the outcome measure, and a 2-sided significance test. Thus, following a 1:2 recruitment ratio, the estimated required sample was 55 participants in the control arm and 110 participants in the intervention arm. The sample size estimate was split into 4 equally sized groups to estimate the number of participants needed to be recruited from each department. Originally, this 2-arm trial had an equal recruitment ratio across 4 departments, including the department of pain management, which planned to recruit patients with both chronic and acute postoperative pain [19]. However, recruitment of patients with postoperative pain ceased unexpectedly in March 2022, and the remaining recruitment was slow. To ensure the necessary statistical power and allow for more variation in the use of the intervention, our alterations prompted a new power analysis, and the follow-up period was reduced from 12 months to 6 months. Details of the power calculation and rationale for the effect size are summarized in the published protocol [19].

This trial was a prospective, unblinded, nonrandomized controlled study. Allocation was conducted sequentially, meaning that patients were allocated to the control arm first, and when the required number was allocated, the participants were recruited to the intervention arm based on the sample size calculation. Allocation was conducted within each department, and when the required number of control participants were recruited, recruitment for the intervention arm began immediately, even if other departments were still recruiting for the control arm. Blinding was not possible because of the nature of the intervention.

Approval for the project was obtained after review by the institutional data protection officer at the University Hospital of North Norway for the cancer department (2021/4942) and by the data protection officer at the Oslo University Hospital for the remaining departments (21/06826). The project protocol was prereviewed by the regional ethics committee and regarded as outside the mandate according to the Norwegian Health Research Act (regional ethics committee southeast reference number 252051). All participants provided informed and written consent. All data were anonymized before analyses, and only the first author (HH) had access to the anonymizing key to safeguard participant information. Patients did not receive any compensation for their participation.

All baseline data were reported as counts and percentages or means and SDs for normally distributed data and medians and ranges for skewed data. To investigate any baseline differences between the arms because of the lack of randomization, all baseline data were compared between the 2 arms using the chi-square test, Student t test (2-tailed), and Mann-Whitney U test, as appropriate. Missing data were not imputed. For all continuous variables, the individual mean change was estimated by subtracting the baseline score from the follow-up score at both 3 months and 6 months. Differences in mean changes between the arms were tested using the Student t test (2-tailed). A post hoc sensitivity analysis was performed using linear regression analysis, adjusting for the observed differences in the baseline characteristics between the arms (history of COVID-19 and domains 1 and 7 in eHLQ). P<.05 was considered statistically significant. Cronbach α was computed to evaluate the internal consistency of the scales based on the assumptions that the responses to individual questions were normally distributed, had equal variance, and equally explained the factor. All analyses were performed using SPSS (version 29; IBM Corp).

During the recruitment period, 162 patients signed the digital informed consent forms and completed consecutive self-reported baseline questionnaires. Given the 1:2 allocation ratio in favor of the intervention arm, we successfully allocated 55 (34%) patients to the control arm and 107 (66%) patients to the intervention arm. The participant flow and discontinuation are depicted in Figure 1. Overall, recruitment and allocation to the control arm started in September 2021, and recruitment to the intervention arm started in January 2022; sufficient participants were recruited by December 2022 (specific dates of the start and end of the recruitment are provided in Multimedia Appendix 2). Recruitment to the 2 arms differed slightly among the 4 departments. The data collection ended in June 2023.

In the control arm, 11% (6/55) of the participants did not respond at 3 months, while among those still alive and included (ie, not with retracted consent) at 3 months, 10% (5/52) of the participants did not respond at 6 months. In the intervention arm, 13.1% (14/107) of the participants did not respond at 3 months, and 6.9% (7/101) of the participants did not respond at 6 months.

The 162 participants included in this study, on average, were aged 52.6 (SD 15.8) years at baseline, with a range of 20 to 83 years (Table 2). There were 90 (56.6%) female participants; most (n=108, 66.7%) participants had a college education or higher, 99 (61.1%) were employed, and 32 (19.8%) were retired. There were no statistically significant differences between the control and intervention arms in terms of demographic or clinical characteristics, except for a history of COVID-19, subsequent fear of COVID-19 (Table 2), and a higher score in the intervention arm in eHLQ domains 1 and 7 (Table 3).

The participants in the intervention arm used an Apple iOS device (52/106, 49.1%) more frequently than an Android operating system (31/106, 29.2%; missing information on system: 23/106, 21.7%). After 3 months, 75% (65/87) of the intervention arm responders self-reported still using the MyDignio app, and after 6 months, 8.6% (91/106) of the participants were categorized as high users. The use of the intervention was reported for the entire 6-month intervention (Table 4). Although patients from the cancer, lung, and neurology departments were expected to use the digital outpatient care throughout the 6 months they were in the trial, the pain department had patients in digital outpatient care for the purpose of both pain medication reduction and pain medication adjustments, typically for a shorter time than 6 months. Thus, patients from the pain department used the app for an average of 11.3 (SD 6.5) weeks, with a median of 9.5 (range 3-27) weeks.

After 3 months, we observed a positive change in health literacy, which was measured using the HLQ (mean 0.08, SD 0.44) in the intervention arm, while a negative change (mean change –0.10, SD 0.51; P=.03) was observed in the control arm (Table 5). Similarly, in the HLQ domain 3, the intervention arm had a positive mean change compared with a negative mean change in the control arm. However, this change was not clinically significant (P<.50).

In HRQoL, a favorable mean change was observed in the intervention arm compared with the control arm for the physical component score (mean difference –3.29; P=.006) and mental component score (mean difference –3.08; P=.02).

For the primary outcome, health literacy after 6 months, which was measured using the HLQ domain 9, we did not observe any statistically significant differences in the scores (mean difference –0.05, 95% CI –0.20 to 0.10; P=.53; Table 6). Accounting for the differences in the history of COVID-19 and baseline eHLQ scores in domain 1 and 7 between the arms did not influence the results (mean difference –0.04; 95% CI –0.13 to 0.21, P=.66). However, in the eHLQ, there was a statistically significant difference in domain 4 regarding feeling safe and in control, where the control arm had a positive change, while the intervention arm had a negative mean change.

There were no differences in general satisfaction with the outpatient service at 6-month follow-up, with 76% (34/45) of the participants in the control arm and 78% (71/91) in the intervention arm being satisfied (P=.42). There were no differences between the arms in terms of the perceived effect of the treatment on their own health; 18% (8/45) of those in the control arm reported a decline in their health compared with 14% (13/91) in the intervention arm, while the remaining participants reported an improvement or no change (P=.60). The proportion of participants stating that a digital outpatient service would affect their feeling of safety was 49% (27/55) of the control participants and 61.7% (66/107) of the intervention participants (P=.09).

The participants in the intervention arm were generally satisfied or had little concern about the digital intervention (Table 7). Technical difficulties were reported by 18.9% (20/106) of the participants at either the 3- or 6-month follow-up.

There were no differences between the 2 arms after 6 months in terms of medication changes or the number of times patients had contacted the outpatient clinic. The intervention arm reported significantly fewer hospital admissions than the control arm (P=.03). The intervention participants reported 37 hospital admissions (mean 0.35, SD 1.1; range 0-7) among 13 (12.1%) of 107 participants, while the control arm reported 25 hospital admissions (mean 0.5, SD 1.05; range 0-5) among 13 (26%) of 50 control participants.

This study reports findings from a study of a novel multicomponent digital health intervention comprising PRO measures, self-monitoring of physiological measures, and asynchronous chat messages among a heterogeneous group of patients in outpatient care. We did not observe any statistically significant differences in the HLQ domain 9, which assesses patients’ understanding of health information well enough to know what to do, even though this study was presumably adequately powered. Our results demonstrate a trend that digital outpatient care, at the 3-month follow-up, improved health literacy domains encompassing patients’ ability to actively manage their own health and whether they understood health information well enough to know what to do. In addition, both physical and mental HRQoL improved after 3 months of digital outpatient care compared with those in the control arm; however, this statistically significant change was not considered clinically significant (P<.50). At the 6-month term, these positive findings were not observed; in contrast, the patients in the intervention arm demonstrated a statistically significant decline in digital/eHealth literacy in the eHLQ domain feel safe and in control. Overall, the participants in digital outpatient care had a high satisfaction rate when evaluating the digital outpatient care platform. Previous research was inconclusive regarding the outcomes of digital health interventions [2-9]. Our study adds to this field by being the first study to explore the associated outcomes of a multicomponent and comprehensive digital outpatient intervention. Despite the noted limitations of our study, some reflections on our findings will be relevant to bring the research field forward.

Several mechanisms may explain our findings. Specifically, we argue that digital outpatient care is positively associated with health literacy and HRQoL because of several mechanisms of the digital care model. First, the use of digital outpatient care allows for the collection of PRO measures on a regular basis, thus increasing patient involvement. Favorable health outcomes related to such involvement have previously been observed [46], although evidence remains limited because of the heterogeneity of research studies in this field [9]. In our study, both the patients and the health care workers had access to a convenient tool for short, asynchronous messages through the digital platform. In line with previous research [47,48], for many of our patients, these asynchronous messages served as a tool to clarify minor questions, provide little information regarding their clinical status and self-management, and give a response without having to wait in a telephone queue. Finally, the department providing care for patients with ILD offered the patients equipment for home spirometry and pulse oximetry. Such home monitoring gave the patients their first experience of truly being able to self-manage their condition from home while data were sent to health care workers to ensure that proper care was provided.

Answering disease-related PRO measures can, in itself, increase reflection regarding self-management and skills in patients [20,49]. Favorable outcomes are also seen among those with the opportunity to ask questions regarding their own condition [47,48]. Altogether, enabling reflection and asking questions in the digital service might be the 2 crucial components supporting the short-term increase in health literacy in our study. Furthermore, limited health literacy has been suggested to predict the increased use of health resources [50]. Conversely, the observed increase in health literacy in our study might reduce the use of health resources over time, confirming previous research that found an association between supporting health literacy and improved self-management [9,14]. We found significantly fewer hospital admissions in the intervention arm, although this finding was not observed for the number of contacts with the outpatient clinic. The experience of increased self-management can also reduce the negative impact of living with a long-term or chronic condition, thus explaining the favorable change in HRQoL. However, as improved health literacy at 3 months was not found at 6 months, some uncertainty remains, and more research is needed to explore the association between health literacy and the use of health service resources.

Although the participants used digital tools to varying extents based on clinical indications, these new services provided favorable short-term outcomes. Short-term changes in favor of an intervention are often demonstrated, while long-term effects are more challenging to observe. Although we found an increase in health literacy and HRQoL after 3 months, we did not observe this increase after 6 months. Instead, we found a negative change in the digital/eHealth literacy domain concerning feelings of safety and control. We cannot fully explain these findings based on our data; however, we believe that the 6-month contact point represents a vulnerable time for patients in a digital outpatient care model. Initially, the new services spark curiosity, foster a sense of increased self-management, and help patients feel more capable of doing the right thing. However, after some time, this may change as patients develop their skills, use more digital care, and simultaneously become more aware of the elements they cannot control. Thus, while they might, in fact, have better control, their increased skills make them more cognizant of the uncertain factors, which creates the feeling of less control that we found. Another mechanism supporting this hypothesis is that despite the lower feeling of control, the participants reported high satisfaction with the digital service after 6 months. Previous research has not observed or discussed potential solutions to such issues. Thus, future research should be aware of the potentially increased need for support among patients during participation in a digital outpatient care model. This is not only to reduce dropout but also to reduce any potential feeling of losing control.

Patients with a persistent need for health services to support their self-management challenge traditional outpatient care, and new ways to provide patient care are warranted to uphold sustainable services of high quality [10]. Most of the participants in our intervention arm used digital outpatient care as intended; that is, they responded to PRO measures, uploaded self-monitoring measures, and asked for clarification when needed. These findings contrast with previous research, which suggests that maintaining participant engagement through a digital intervention can be challenging [51], even through a life course with chronic or long-term conditions [52,53]. Factors that are important for ensuring the use of digital care include a referral from a health care worker and an interest among both the patients and the health care workers in digital services [54]. In our study, we observed a low dropout rate and high satisfaction, suggesting that the patients were content with the study and the digital outpatient care they received. Whether the use and satisfaction of digital outpatient care would have been the case in a longer term beyond 6 months was beyond the scope of our study and should be investigated in future research. However, given how well the digital outpatient care was received and used by the patients, we suggest that models such as ours should be investigated further to explore their relevance and effects on clinical practice.

This study has some limitations, several of which were discussed in the published protocol [19]. First, there is a risk of selection bias because of the lack of randomization into either a control or intervention arm. However, notably, because the allocation was performed sequentially (first for the control arm and then for the intervention arm), there was no bias directly due to patient preferences. Moreover, the 2 arms were similar at baseline except for the history of COVID-19 and subsequent fear of COVID-19 and eHLQ domains 1, using technology to process health information, and 7, digital services that suit individual needs. The control arm was mainly recruited when very few people had contracted COVID-19, while the intervention arm was recruited when the strategy had shifted because of vaccinations and more people contracting COVID-19. As a result, the timeline of the COVID-19 pandemic and eHLQ may have caused some bias in this study. However, sensitivity analysis adjusted for baseline differences provided similar findings.

Second, we do not know how many patients were assessed for eligibility because these data were not sufficiently systematized during enrollment. We had to alter recruitment during the trial from a 1:1 allocation to a 1:2 allocation because of slow recruitment [19]. However, the revised power calculation with a sequential 1:2 allocation provided a sufficient sample size as anticipated a priori. This study was not powered to detect clinical differences within each of the 4 patient groups (ie, from the pain, lung, cancer, and epilepsy departments) or in any of the secondary objectives. Because of the many tests performed, false positive findings might have occurred, so the results should be interpreted with caution. Furthermore, we cannot provide evidence of the findings of digital outpatient care on disease-specific clinical parameters. Rather, we suggest that future research investigate whether our observed changes in health literacy affect disease-specific clinical or health outcomes over time, as has been observed in other studies [13,55].

This study explored digital outpatient care comprising PRO measures; asynchronous messaging; and remote monitoring of clinical indications for patients with chronic pain, ILD, epilepsy, or cancer. No significant differences were observed between the 2 arms in our primary outcome, assessing patients’ health literacy reflected through their understanding of health information after 6 months. Our data indicate an improvement in some health literacy domains and HRQoL at 3 months among the patients using the digital solution. Despite our mixed results, the participants reported high satisfaction with the digital outpatient care intervention; therefore, digital health interventions can be positive for people with chronic conditions and are well received. However, our findings are not entirely consistent, and more research should explore the mixed changes in health or eHealth literacy, the effectiveness of such interventions on various outcomes, and the relationship between participant satisfaction and clinical effects.