JMIR J Med Internet Res Journal of Medical Internet Research 1438-8871 JMIR Publications Toronto, Canada v27i1e63639 40324173 10.2196/63639 Review Review Electronic Implementation of Patient-Reported Outcome Measures in Primary Health Care: Mixed Methods Systematic Review Sarvestan Javad Leung Tiffany Peeples Malinda He Wan-jia Sasseville Maxime PhD 1
Faculty of Medicine Université Laval Quebec City, QC Canada 1 877 785 2825 maxime.sasseville@fsi.ulaval.ca
https://orcid.org/0000-0003-1694-1414 Supper Wilfried PhD 1 https://orcid.org/0009-0002-6642-5768 Gartner Jean-Baptiste PhD 1 https://orcid.org/0000-0001-5907-6112 Layani Géraldine MSc, MD 2 https://orcid.org/0000-0003-1764-6433 Amil Samira MSc, RD, MD 1 https://orcid.org/0000-0002-4024-9762 Sheffield Peter MSW, MSc 3 https://orcid.org/0009-0009-1869-5035 Gagnon Marie-Pierre PhD 1 https://orcid.org/0000-0002-0782-5457 Hudon Catherine PhD, MD 4 https://orcid.org/0000-0001-6140-9916 Lambert Sylvie RN, PhD 5 6 https://orcid.org/0000-0002-9202-9251 Attisso Eugène MSc, MD 1 https://orcid.org/0000-0002-0820-0993 Ouellet Steven PhD 1 https://orcid.org/0000-0003-2158-0043 Breton Mylaine PhD 4 https://orcid.org/0000-0001-5713-9618 Poitras Marie-Eve RN, PhD 4 https://orcid.org/0000-0002-3315-0190 Roux-Lévy Pierre-Henri MD, PhD 4 https://orcid.org/0009-0002-7716-9931 Plaisimond James MA 1 https://orcid.org/0009-0007-4986-7712 Bergeron Frédéric MSI 1 https://orcid.org/0000-0003-0978-7420 Ashcroft Rachelle PhD 3 https://orcid.org/0000-0002-5666-1946 Wong Sabrina T. RN, PhD 7 https://orcid.org/0000-0002-9619-9012 Groulx Antoine MSc, MD 1 https://orcid.org/0000-0002-0995-093X Paquette Jean-Sébastien CCMF, MSc, MD 1 https://orcid.org/0000-0002-9524-6761 D'Anjou Natasha 8 https://orcid.org/0009-0002-7168-6639 Langlois Sylviane 8 https://orcid.org/0009-0004-5074-518X LeBlanc Annie PhD 1 https://orcid.org/0000-0002-5377-8102 Faculty of Medicine Université Laval Quebec City, QC Canada Faculty of Medicine Université de Montréal Montréal, QC Canada Factor-Inwentash Faculty of Social Work University of Toronto Toronto, QC Canada Faculty of Medicine and Health Sciences Université de Sherbrooke Sherbrooke, QC Canada Faculty of Nursing Université McGill Montréal, QC Canada Centre de recherche de St. Mary's Montréal, QC Canada UBC Centre for Health Services and Policy Research and School of Nursing Vancouver, BC Canada Patient Partner Université Laval Québec, QC Canada Corresponding Author: Maxime Sasseville maxime.sasseville@fsi.ulaval.ca 2025 5 5 2025 27 e63639 27 6 2024 20 1 2025 9 2 2025 3 3 2025 ©Maxime Sasseville, Wilfried Supper, Jean-Baptiste Gartner, Géraldine Layani, Samira Amil, Peter Sheffield, Marie-Pierre Gagnon, Catherine Hudon, Sylvie Lambert, Eugène Attisso, Steven Ouellet, Mylaine Breton, Marie-Eve Poitras, Pierre-Henri Roux-Lévy, James Plaisimond, Frédéric Bergeron, Rachelle Ashcroft, Sabrina T. Wong, Antoine Groulx, Jean-Sébastien Paquette, Natasha D'Anjou, Sylviane Langlois, Annie LeBlanc. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 05.05.2025. 2025

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

 Background

Managing chronic diseases remains a critical challenge in primary health care (PHC) across the Organization for Economic Co-operation and Development countries. Electronic patient-reported outcome measures (ePROMs) are emerging as valuable tools for enhancing patient engagement, facilitating clinical decision-making, and improving health outcomes. However, their implementation in PHC remains limited, with significant variability in effectiveness and adoption.

 Objective

This systematic review aimed to assess the implementation and effectiveness of ePROMs in chronic disease management within PHC settings and to identify key barriers and facilitators influencing their integration.

 Methods

A mixed methods systematic review was conducted following the Cochrane Methods and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. We included studies that implemented ePROMs among adults for chronic disease management in PHC. The extracted data included patient health outcomes, provider workflow implications, implementation factors, and cost considerations. The reach, effectiveness, adoption, implementation, and maintenance framework was used.

 Results

Our search yielded 12,525 references, from which 22 (0.18%) studies were included after screening and exclusions. These studies, primarily conducted in the United States (n=9, 41%) and Canada (n=8, 36%), covered various chronic diseases and used diverse ePROM tools, predominantly mobile apps (n=9, 41%). While some studies (n=10, 45%) reported improvements in patient health outcomes and self-management, others (n=12, 55%) indicated no significant change. Key barriers included digital literacy gaps, integration challenges within clinical workflows, and increased provider workload. Facilitators included strong patient-provider relationships, personalized interventions, and technical support for users. While some studies (n=10, 45%) demonstrated improved patient engagement and self-management, long-term cost-effectiveness and sustainability remain uncertain.

 Conclusions

Success in implementing ePROMs in PHC appears to hinge on addressing digital literacy, ensuring personalization and meaningful patient-provider interactions, carefully integrating technology into clinical workflows, and conducting thorough research on their long-term impacts and cost-effectiveness. Future efforts should focus on these areas to fully realize the benefits of digital health technologies for patients, providers, and health care systems.

 Trial Registration

PROSPERO CRD42022333513; https://www.crd.york.ac.uk/PROSPERO/view/CRD42022333513

 International Registered Report Identifier (IRRID)

RR2-10.2196/48155

 patient-reported outcomes measures digital health implementation science systematic review PRISMA Introduction

The World Health Organization advocates for universal access to primary health care (PHC), a critical first point of contact within health care systems in the Organization for Economic Co-operation and Development countries, despite ongoing access limitations [1-4].

Chronic diseases, primarily managed in this setting, continue to be leading causes of mortality and morbidity in these countries [5,6]. The Canadian Institute of Health Research promotes the use of patient-reported outcome measures (PROMs) to enhance patient experience, clinical outcomes, and health care efficiency [7]. While PROMs have been widely implemented in hospital settings, their use in primary care remains underexplored. PROMs are assessments of a patient’s health status based on their own perceptions, without input from a third party. These reports are collected using validated questionnaires that quantify aspects such as quality of life, disease management, daily functioning, and symptoms [8]. For more than a decade, governments in several countries have funded initiatives to develop, implement, and use PROMs in hospitals [9] and PHC settings [10]. Optimal implementation and use of PROMs are associated with clinical benefits, such as improved communication with patients, better adaptation of health care to patient needs, and shorter consultation times [9].

The emergence of digital tools has a great potential for the implementation and use of PROMs in health care settings [11]. Digital methods, compared to traditional pen-and-paper approaches, enhance data collection quality, reduce costs, support clinical decision-making, and are better received by patients [12]. Several systematic reviews have identified the barriers and facilitators associated with their implementation in health care systems and evaluated the effectiveness of digitally collected electronic PROMs (ePROMs) [8,13,14]. In oncology, the main barriers to implementation are increased workload and inadequate technological infrastructure [8,14]. In terms of effects, these systematic reviews note that there is a fairly wide divergence between the results of the studies that have been identified. However, in oncology and pediatrics, the implementation of ePROMs is associated with an increase in quality of life and patient satisfaction [8,13]. While systematic reviews have identified impacts, barriers, and facilitators for ePROMs implementation in specialized settings, these findings have limited transferability to PHC due to differences in patient populations and clinical workflows; for example, the wider age range in primary care makes adapting ePROMs to varying levels of digital literacy more challenging. The implementation of ePROMs in PHC has however the potential to provide unique benefits, particularly by supporting self-management of chronic disease symptoms [15].

This review aims to evaluate the implementation and effectiveness of ePROMs for chronic disease management in PHC, identifying associated barriers and facilitators.

 Methods Ethical Considerations

We conducted a systematic review of the literature according to the Cochrane Methods Group and in compliance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines for its reporting (Multimedia Appendix 1) [16,17]. We registered the study protocol with the PROSPERO Systematic Review Registry (ID: CRD42022333513) [18] and have also published it [19].

 Synthesis Questions

The synthesis questions were (1) What are the effective strategies to implement ePROMs in PHC? (2) What are the challenges and barriers and facilitators to successful implementation of ePROMs in PHC? and (3) What are the outcomes of ePROMs in PHC chronic disease management?

 Eligibility Criteria

We included all types of evidence matching the following PICOS (Population, Intervention, Comparison, Outcomes, and Setting) [16]:

Population: All studies including the implementation of an ePROM among adults for chronic disease management

Interventions or phenomena of interest: No restrictions. We included all types of implementations, theoretical models, structures, or PROMs.

Comparator: No restrictions

Outcomes: We considered all outcomes reported in the studies. We sought outcomes related to patients, caregivers, health care providers, policy makers, barriers, facilitators, acceptability, feasibility, adoption, fidelity, morbidity, mortality, quality of life, satisfaction, cost, and cost-effectiveness.

Setting: We included studies conducted exclusively in PHC settings, regardless of location, and extracted information on implementation.

We included all types of empirical studies (qualitative, quantitative, and mixed methods) published in French, English, or Spanish.

 Search Strategy

Using an iterative process, the search strategy was developed in collaboration with an experienced information specialist (FB). On August 15, 2022, we searched the following databases: MEDLINE (OVID), Embase (via Embase), CINAHL (EBSCO), and Web of Science. Considering the large number of results, we decided not to consult gray literature sources (eg, GreyNet, Grey Matters, Google, websites, and ProQuest Dissertations & Theses), as suggested in our protocol. We applied no restrictions to the search strategy, including time limit, as mentioned in our protocol due to the lack of effect in reducing the number of results. The full search strategy is available in Multimedia Appendix 2.

 Data Collection and Screening

We exported all citations in the web-based collaboration tool Covidence, where duplicates were removed with the automated function [20]. Pairs of reviewers screened the titles and abstracts independently. We retained ambiguous or incomplete abstracts to be reviewed in full. We searched and obtained all the full texts of the selected references and imported the PDF files in Covidence. Pairs of reviewers then independently applied the inclusion criteria using the full texts following a pilot testing using the process outlined above. At any moment in the screening process, the first author helped resolve any discrepancy. All the reasons for exclusions were recorded in Covidence, and a PRISMA flowchart describes study identification, screening, inclusions, and exclusions (Figure 1) [21].

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram.

 Data Extraction and Appraisal

We extracted descriptive data (title, year of publication, authors, funding, conflicts of interests, and country), study types (published or gray literature), methodological data (design, sample size, measure constructs, and name of the instrument), setting data (clinical setting, type health professionals, and patient population), characteristics of the ePROM tools (functionality, interface and delivery platform, and integration approach), implementation data (description of implementation strategies, facilitators, and barriers), outcomes (patient health, providers workflow, and cost), and outcomes type (qualitative and quantitative). The quality of the included studies was evaluated using the Mixed Methods Appraisal Tool (MMAT), which is designed for systematic reviews that synthesize data from qualitative, quantitative, and mixed methods studies [22].

 Data Synthesis

We used the reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework as a data analysis framework. RE-AIM has been developed to evaluate the public health impacts of interventions and has been used in systematic reviews to help structure the assessment of the different implementation factors at play in complex contexts and settings [23,24]. This framework includes 5 dimensions: reach (how willing the targeted population is to participate in the intervention; ie, ePROMs), efficacy (what is the impact of the intervention on outcomes), adoption (can this be adopted by new groups with ease and minimal changes), implementation (what are the special issues and barriers), and maintenance (can the intervention be maintained and the impact continued). The use of RE-AIM enabled us to give an overview of the parameters strengthening (review questions 1 and 2) the efficiency (review question 3) of ePROMs’ integration in PHC and its impact on outcomes.

We used a 2-phase sequential mixed methods synthesis design, that is, conduct a qualitative synthesis and use its results to inform the quantitative synthesis [25]. For phase 1 (qualitative), we summarized and described methods and approaches designed to implement and integrate ePROMs in PHC, using a thematic synthesis procedure [26]. The qualitative data synthesis produced narrative summaries of main themes, which were then classified according to the RE-AIM framework. We summarized study characteristics and methodological differences and similarities to highlight the following points: strengths and weaknesses of each implementation method, main outcomes of implementation, main resources used and their impacts, and if any trade-offs are described and their effect on the results of the study.

 Results Overview

The PRISMA diagram (Figure 1) shows the results of the search strategy and study selection process. Our search strategy identified 12,525 references. The exclusion of duplicates (n=3998) and the first selection stage (titles and abstracts) led us to retain 761 references. A further 749 references were excluded when the texts were read in full. In all, 22 studies met the selection criteria and were therefore selected.

 Study Characteristics

Table 1 summarizes descriptive data from the 22 selected studies; of these, 5 (23%) are mixed-design studies, 6 (27%) are qualitative studies, and 11 (50%) are quantitative studies. These 22 studies were published between 2015 and 2022, and the majority were conducted in the United States (n=9, 41%) and Canada (n=8, 36%). The number of participants included in these studies ranged between 2334 and 8, and only 3 studies had >300 participants. The aims of the studies selected align around improving clinical outcomes for patients with chronic diseases and enhancing health care delivery through better integration of patient-reported data into clinical practice. In most studies, the average age of the population was >50 years. Gender representation varied from study to study, but in the majority (16/22, 73%), women represented >50% of participants. Additional information about the intervention are presented in Multimedia Appendix 3.

The main chronic diseases population targeted are asthma (4/22, 18%), cardiovascular diseases (3/22, 14%), multimorbidity (5/22, 23%), mental health (3/22, 14%), diabetes (2/22, 9%), or complex cases of chronic disease (2/22, 9%). Regarding ePROMs, the digital tool most frequently implemented was a mobile app (9/22, 41%). The constructs of ePROMs varied between 3 main categories: general health (eg, quality of life and health status), symptom monitoring (eg, for sleep, pain, anxiety, and depression), and self-management (eg, self-efficacy and patient activation).

Studies description.

Study Country Aim of the study Age (y), mean Women (%) Targeted Constructs Chronic diseases Sample size, n Study designa
Staeheli et al [27] United States Compare screening results to data derived from chart reviews of patients seen before the deployment of the screening intervention to determine the following:

The rates of unrecognized and undiagnosed depression, PTSDb, and risky drinking (referred to collectively as “behavioral health problems” for the purposes of this study) in this patient population and

Whether increased recognition of behavioral health problems in the encounter was associated with appropriate treatment and follow-up of identified patients.

 Not specified but around 50 66.54 Depression, PTSD, and problem drinking PTSD, depression, and risky drinking 275 Quantitative
Ainsworth et al [15] United Kingdom

Assess the feasibility of a trial to evaluate a digital intervention in primary care to improve quality of life and other clinical outcomes of people with asthma, in comparison to usual care

 		56.6 53 Quality of life and self-monitoring Physician-diagnosed asthma 88 Quantitative
Harle et al [28] United States

Assess whether integrating PROsc data in an EHRd affects providers and patient satisfaction with chronic noncancer pain care

 		43.8 62 Pain interference, pain behavior, fatigue, and anger Chronic musculoskeletal 680 Quantitative
Kroenke et al [29] United States

Assess the effectiveness of providing patient-reported outcomes measurement information system symptom scores to clinicians on symptom outcomes

 		49 72 SPADEe SPADE symptom 256 Quantitative
Lear et al [30] Canada

To compare the effect of an internet-based self-management and symptom monitoring program targeted to patients with ≥2 chronic diseases (internet chronic disease management) with usual care on hospitalizations over a 2-y period

 		70.5 38.40 Self-monitoring and self-management ≥2 chronic conditions 229 Quantitative
Miranda et al [31] Canada

Estimate the cost of an ePROf tool and examine whether the benefits gained from the tool outweighed its costs from the perspective of Canada’s publicly funded health care system

 		Not specified but they recruiter patients ≥60 years old Not specified Global health ≥2 chronic conditions 45 Quantitative
Baron and Duffecy [32] United States

Evaluate the feasibility and preliminary efficacy of a technology-assisted sleep extension intervention among individuals with prehypertension or stage 1 hypertension on sleep, blood pressure, and PROs.

 		45.8 50 Sleep quality 24 h ambulatory blood pressure 16 Quantitative
Owen-Smith et al [33] United States

Describe the infrastructure designed to automate PRO data collection

Compare study-enhanced PRO completion rates to those in clinical care

Assess patient response rates based on the PRO administration method and their sociodemographic or clinical characteristics.

 		Not specified Not specified Pain Chronic pain 632 Quantitative
Ramallo-Fariña et al [34] Spain

Assess the effectiveness of different interventions of knowledge transfer and behavior modification to improve the PROMsg of patients with T2DMh in the long term

 		55.7 51.90 Diabetes empowerment Type 2 diabetes mellitus at least 1 y before 2334 Quantitative
Tamisier et al [35] France

Assess the effect of a multimodal telemonitoring intervention on treatment adherence, quality of life, and functional status in symptomatic patients with OSAi and low cardiovascular risk

 		50.6 63.59 Quality of life OSA 206 Quantitative
Trick et al [36] United States

Evaluate time burden for patients and factors associated with response times for an audio computer-assisted self-interview system integrated into the clinical workflow

 		57.0 58 General health Chronic illness 1670 Quantitative
Yanicelli et al [37] Argentina

Validate a home telemonitoring system for the first time in a real setting; its effectiveness to improve self-care and treatment adherence of patients with HFj was assessed through a randomized controlled clinical trial in Argentina

 		52 20 Weight, blood pressure, heart rate, and symptoms HF 30 Quantitative
Bezerra Giordan et al [38] Australia

Explore the patient’s and primary care clinician’s perspective on the facilitators and barriers to using mobile apps, as well as desired features, to support HF self-management

 		69 66.66 Self-management Confirmed diagnosis of HF 12 Qualitative
Steele Gray et al [39] Canada

Tested the usability and feasibility of adopting the ePRO tool into a single interdisciplinary primary health care practice.

The Fit between Individuals, Fask, and Technology framework was used to guide our assessment and explore whether the ePRO tool is feasible for adoption in interdisciplinary primary health care practices and usable from both the patient and provider perspectives

 		58 54.54 General Health Scale and pain ≥2 chronic conditions 14 Qualitative
Hans et al [40] Canada

How the ePRO mobile app and portal system, designed to capture patient-reported measures to support self-management, affected primary care provider workflows

 		56.3 50 Global health and pain interference ≥2 chronic conditions 18 Qualitative
Harle et al [28] United States

Identify the lessons learned in overcoming barriers to collecting and integrating PROs in an EHR

 		Not specified Not specified Pain interference, pain behavior, fatigue, and physical function Chronic pain 12 Qualitative
Irfan Khan et al [41] Canada

Explore the experience and expectations of patients with multimorbidity and their providers around the use of the ePRO tool in supporting self-management efforts

 		58 50 Global health, pain interference, and generalized anxiety Multimorbid 9 Qualitative
Schoenthaler et al [42] United States

Apply a systematic, user-centered design approach to develop i-Matter (investigating a mobile health texting tool for embedding patient-reported data into diabetes management), a theory-driven, mobile PRO system for patients with T2D and their primary care providers.

 		62.5 67 Diabetes self-management T2D for ≥6 mo 12 Qualitative
Steele Gray et al [43] Canada

What are the contexts, processes, and outcomes most relevant to the ePRO intervention?

What are the central (critical to achieving outcomes) versus peripheral (less critical and potentially context dependent) mechanisms that underpin the content theory of the ePRO intervention?

 		Not clear but around 63 68.75 Goal attainment Not specified 16 Mixed methods
Ahmed et al, 2021 [44] Canada

Identify the potential barriers and enablers to using PROMs in primary care LBPk clinical practice from the perspective of health care team members

Develop a theory-based tailored knowledge translation intervention to facilitate the use of PROMs in interdisciplinary clinical practice

 		39 39 Not specified LBP 18 Mixed methods
Steele Gray et al [45] Canada

Evaluate the implementation and effectiveness of the ePRO mobile app and portal system designed to enable goal-oriented care delivery in interprofessional primary care practices

 		68.7 65.22 Goal setting, self-management, mental health, and social health Older adults with complex needs 44 Mixed methods
Bauer et al [46] United States

Assess the feasibility and acceptability of a mobile health platform supporting collaborative care

 		Not specified but around 35 59 Depressive and anxiety symptoms Depression or anxiety disorder 17 Mixed methods

aQualitative: studies primarily collecting and analyzing nonnumerical data (eg, interviews and focus groups), quantitative: studies primarily collecting and analyzing numerical data (eg, randomized controlled trials and cohort and cross‐sectional studies), and mixed methods: studies integrating both quantitative and qualitative approaches. We also reference these criteria in the Data Extraction and Appraisal section for clarity.

bPTSD: posttraumatic stress disorder.

cPRO: patient-reported outcome.

dEHR: electronic health record.

eSPADE: sleep problems, pain, anxiety, depression, and low energy or fatigue.

fePRO: electronic patient-reported outcome.

gPROM: patient-reported outcome measure.

hT2DM: type 2 diabetes mellitus.

iOSA: obstructive sleep apnea.

jHF: heart failure.

kLBP: lower back pain.

 Assessment of Studies’ Methodological Quality

The evaluation of the studies’ methodological quality shows that the majority (15/22, 68%) received a score of ≥80%. Specifically, 38% (8/22) of the studies scored 100%, 33% (7/22) of the studies scored between 80% and 95%, and 19% (4/22) of the studies scored between 60% and 75%. Only 2 studies scored ≤40% (Table 2). While most of the included studies (15/22, 68%) met a high standard of methodological rigor (≥80% on the MMAT), this does not entirely eliminate concerns about heterogeneity, smaller sample sizes, and diverse outcome measures. Higher-quality studies generally provided robust justifications for their choice of ePROM tools and specified clear implementation processes (eg, training sessions and technical support). In contrast, studies with lower MMAT scores often lacked detail on how they integrated ePROMs into existing workflows or offered minimal information on training protocols. These distinctions may influence both the internal validity and generalizability of the findings.

Assessment of studies’ methodological quality using the Mixed Methods Appraisal Tool (MMAT).

Study MMAT items
1.1 1.2 1.3 1.4 1.5 2.1 2.2 2.3 2.4 2.5 3.1 3.2 3.3 3.4 3.5 5.1 5.2 5.3 5.4 5.5
Staeheli et al [27] —a — — — — — — — — — 1b 1 1 0c 1 — — — — —
Bauer et al [46] 1 1 1 1 1 — — — — — 0 1 0 0 0 1 1 1 1 1
Trick et al [36] — — — — — — — — — — 1 1 0 1 1 — — — — —
Owen-Smith et al [33] — — — — — — — — — — 0 1 1 0 1 — — — — —
Steele Gray et al [43] 1 1 1 1 1 1 0 1 0 1 — — — — — 1 1 1 0 0
Bezerra Giordan et al [38] 1 1 1 1 1 — — — — — — — — — — — — — — —
Schoenthaler et al [42] 1 1 1 1 1 — — — — — — — — — — — — — — —
Ahmed et al [44] 1 1 0 1 1 — — — — — — — — — — — — — — —
Kroenke et al [29] — — — — — 1 1 1 0 1 — — — — — — — — — —
Lear et al [30] — — — — — 1 1 1 1 1 — — — — — — — — — —
Baron et al [32] — — — — — 0 0 0 0 1 — — — — — — — — — —
Ramallo-Fariña et al [34] — — — — — 1 1 1 0 1 — — — — — — — — — —
Tamisier et al [35] — — — — — 1 0 1 1 1 — — — — — — — — — —
Yanicelli et al [37] — — — — — 1 0 0 0 1 — — — — — — — — — —
Ainsworth et al [15], 2019 — — — — — 1 1 1 1 1 — — — — — — — — — —
Steele Gray et al [45] 1 1 1 1 1 1 1 1 1 1 — — — — — 1 1 1 0 1
Steele Gray et al [39] 1 1 1 1 1 — — — — — — — — — — — — — — —
Hans et al [40] 1 1 1 1 1 — — — — — — — — — — — — — — —
Harle et al [28] 1 1 1 1 1 — — — — — — — — — — — — — — —
Harle et al [28] — — — — — 1 0 1 0 1 — — — — — — — — — —
Irfan Khan et al [41] 1 1 1 1 1 — — — — — — — — — — — — — — —

aNot applicable.

bThis criterion was met.

cThis criterion was not met.

 Qualitative Results Overview

We present the entire thematic organization and representatives quotes of the qualitative studies structured according to the RE-AIM model in Table 3.

Qualitative themes of the barriers and facilitators according to the reach, effectiveness, adoption, implementation, and maintenance model.

Study Reach, effectiveness, adoption, implementation, and maintenance Actor Categories Themes Representative quotes
Barriers
Steele Gray et al [39] Reach Patient Communication Lack of patient-provider interaction “However, some patients reported feeling isolated with the mobile device, and felt that the tool could become a replacement for in person consultation.”
Bezerra Giordan et al [38] Reach Patients, clinicians, and organizations Digital literacy Lack of digital literacy of patients and providers’ lack of training “Patients mentioned seeing themselves as not being tech-savvy enough to use an app [mostly due to a perceived age barrier] and expected it would take them a long time and effort to learn how to use it properly.”
Irfan Khan et al [41] Efficacy Patient Communication Lack of feedback from providers “Patients identified gaps in the tool’s ability to promote self-efficacy in terms of the adherence to self-regulation activities because of limited feedback on their progress from providers.”
Harle et al [28] Efficacy Providers Treatment Take time away from treatment “Discussing these issues (psychological data on depression and anxiety) could harm care quality by diverting their attention away from acute problems that they judged to be more relevant at a given visit or more aligned with their clinical expertise.”
Steele Gray et al [39] Efficacy Providers Workload Increase workload “Finally, some providers felt that they may be liable for monitoring patients during out-of-office hours, which would require additional time and resources.”
Irfan Khan et al [41] Adoption Patients, clinicians, and organizations Adaptation Reticence to adopt new practices, lack of personalization, and lack of alignment with clinical reality “In addition, patients stressed the need for greater personalization and customizability of goals and monitoring protocols. The questions in the ePRO tool appeared to lack the depth that was considered vital to incorporating patient context into self-management activities.”
Schoenthaler et al [42] Implementation Patient ePROMa digital solution Confidentiality, difficult data entry, difficult use, retrieval of information, data presentation, technical issues, and no technical support “Several patients had difficulty reading the bar graphs of PROs that were collected biweekly (eg, quality of life) and recommended changing the items to weekly measures to be consistent with other PROs.”
Steele Gray et al [39] Implementation Patients, clinicians, and organizations Workflow Workflow integration and data overload “After the initial visit, provider participants reported experiencing difficulty incorporating patient data into their workflow in terms of: (1) increased charting time required to input data into the provider’s EMR and (2) being able to view data in manageable chunks.”
Bauer et al [46] Maintenance Patient Adaptation Lack of personalization of system features and sensitivity to change of PROMb “Patients’ desire for more personalized features. Patients wanted to customize the app to meet their individual needs, for example, by adjusting the timing of prompts, the types of symptoms they were reporting on, or the frequency or content of health tips.”
Irfan Khan et al [41] Maintenance Patient Relationship Lack of patient-provider interaction “Patient expectations around social facilitation and social support indicated that they were expecting a more active role in self-management efforts from providers. Patients felt the tool should supplement patient-provider interaction through regular feedback and encouragement as an ‘add-on’ to existing in-person appointments rather than a replacement for in-person interaction and consults with their providers.”
Bezerra Giordan et al [38] Maintenance Patients, clinicians, and organizations Burden of treatment Increased treatment burden “Participants mentioned the use of an app could be associated with an increase in the burden of managing heart failure. learning how to use the app and remembering to use it could be seen as an additional responsibility in people’s already busy lives, which could be demotivating and lead to a lack of interest and decreased willingness to use the app over time.”
Hans et al [40] Maintenance Providers Workload Increase workload “Providers questioned whether the app would actually improve workflow functions or simply add another task. Multiple providers expressed their concerns with incorporating the ePRO into their daily visit routine.”
Facilitators
Steele Gray et al [45] Reach Providers and patient Relationship Existing patient-provider relationship “The meaningfulness of the ePRO tool was reliant to strong relationships between patients and providers (enabling collective action)”
Irfan Khan et al [41] Reach All participants Digital literacy Digital training and digital literacy “The technology partner (QoC Health) offered providers two 1-hour hands-on training sessions (facilitated by the research team) on the mobile phone app and portal to provide a walk-through of the ePRO tool before starting the study, whereas, patients received one-on-one training through a 30-minute hands-on session with a member of the research team at the time when patients gave consent to participate in this study.”
Ahmed et al [44] Reach Providers ePROM literacy Score interpretation “Participants agreed that having the knowledge and skills to interpret PROM scores is required for clinicians to be able to use PROMs in clinical care for management of LBP.”
Irfan Khan et al [41] Efficacy Patient and providers Self-management Improved self-management, self-efficacy, symptoms management, goal setting, and treatment quality “Patients acknowledged the potential of the ePRO tool in building capacity to support self-management in a team-based care environment by helping to better distribute the workload across providers to meet the evolving needs of patients.”
Bezerra Giordan et al [38] Efficacy Patients, providers, and organizations Communication Interactivity, timely action, team problem-solving, goal setting, patient-provider communication, and patient-oriented treatment “Participants mentioned that the ability for both patients and clinicians to monitor signs of deterioration allowed for timely action.”
Schoenthaler et al [42] Efficacy Providers ePROM digital solution Data presentation “Providers want PRO data that are specific and actionable and can help them focus the clinic visit on what is most important for their T2D patients’ care.”
Irfan Khan et al [41] Efficacy Providers Treatment Treatment quality “Providers emphasized the value of the ePRO tool in helping to generate insights into underlying patient context (ie, patient preferences and readiness) to offer a fulsome sense of how patients are coping, and thereby adjust goals and self-management activities as needed.”
Hans et al [40] Efficacy Providers Workload Reduced resource use and time saved in data retrieval during encounters “Providers reported that the app presented an additional resource that they could leverage to quickly orient themselves to their patients’ wellbeing.”
Schoenthaler et al [42] Adoption Patient and providers ePROM digital solution Availability of the data collection, Interoperability, and data presentation for decision-making “These included defining a threshold that patients’ data can fall above or below and depicting it in a way that makes it easily detectible and depicting it in a way that makes it easily detectible and actionable, using bar graphs to show directionality, including icons or coloring schemes in addition to PRO labels that enhance the readability of the report, and, including summary data in percentages or raw numbers to show the patient’s progress over time.”
Bezerra Giordan et al [38] Adoption Patient Introduction Introduction after an exacerbation “However, this barrier could be overcome by introducing the app soon after an exacerbation, when they might be more willing to improve their self-management practices.”
Schoenthaler et al [42] Implementation Providers Communication Patient-oriented treatment “Similar to patients, they felt that the insight messages were helpful for interpreting the data and prompting behavioral changes.”
Schoenthaler et al [42] Implementation Providers and patient ePROM digital solution Data presentation, collection of only relevant patient data, and comprehensible user interfaces “Patients preferred layouts that used darker fonts and lighter background colors to help make the text easier to read. All patients viewed the color-coded schema favorably because it helped draw attention to the most important aspects of the report and made the data easy to interpret.”
Schoenthaler et al [42] Implementation Providers and patient Psychometrics Valid PROM and sensitivity to change “PROs should show variability in patients’ responses over time and be actionable by both patients and providers.”
Steele Gray et al [39] Maintenance Providers Adaptation ePROMs process not aligned with clinical reality “To improve effectiveness (and efficiency), providers wanted the tool to fit better with their existing workflows and programs, for example, through better alignment with creation of SMART goals for patients or allowing for monitoring protocols that aligned with goals of existing chronic disease management programs.”
Steele Gray et al [39] Maintenance Patient and providers Communication Peer support, feedback aligned with needs, and automated personalized milestones “Tool-enabled feedback, particularly from peers, was also viewed to offer encouragement on progress toward goal attainment by way of a shared experience.”
Steele Gray et al [39] Maintenance Providers and patient ePROM digital solution Longitudinal training, patient prompting, technical support, and interoperability “The meaningfulness of the ePRO tool was reliant to consistent positive assessments of the tool’s utility (regular reflexive monitoring).”

aePROM: electronic patient-reported outcome measure.

bPROM: patient-reported outcome measure.

 Digital Literacy and Training

The data highlight challenges with the lack of digital literacy among patients and providers, alongside insufficient training for providers [38,43]. These issues affected various aspects of the RE-AIM framework, notably impacting reach, adoption, and implementation. Addressing digital literacy through targeted training sessions for both patients and providers emerges as a facilitator by enhancing the usability and acceptance of ePROMs tools [39,41].

 Patient-Provider Communication and Relationship

The dynamics of communication and the relationship between patients and providers are identified as both barriers and facilitators [41,45]. The lack of patient-provider interaction and inadequate feedback are significant barriers, while strong relationships, interactivity, and timely communication facilitate implementation [41].

 Personalization and Integration Into Clinical Workflow

The necessity for ePROMs tools to be personalized and seamlessly integrated into clinical workflows is a common theme across barriers and facilitators. Studies showed the reluctance to adopt new practices due to personalization and integration issues [38,41], contrasting with facilitators that advocate for straightforward data presentation and interoperability [42].

 Technical Challenges and Support

Studies identified technical difficulties, including data entry, information retrieval, and a lack of technical support, as significant barriers [39]. Facilitators highlighted in the studies were ongoing technical support and user-friendly interfaces [47].

 Workload and Treatment Quality

Concerns related to the impact of ePROMs on workload and treatment quality are highlighted barriers [40,43]. The increased workload for providers and potential distractions from acute health issues pose major barriers, while the potential for improved treatment quality through enhanced patient contexts offers a significant advantage [45].

 Quantitative Results Overview

We present an overview of all quantitative results in Table 4.

Quantitative results.

Study Intervention and outcome Effect of the intervention Results Sample size
Steele Gray et al [43] Adopting the ePROa tool on:

Goal Tracker

Health Status Scales and Outcome Measures

 		Null

No statistical differences were seen for pre- versus post means of overall or subscale scores of assessment of QoLb, patient assessment of chronic illness care, and patient activation measure in both the control and intervention groups, nor between control and intervention.

 		16
Staeheli et al [27] Screening intervention on:

Rates of unrecognized and undiagnosed mental health, depression, posttraumatic stress disorder, and risky drinking

 		Positive effect

This tablet-based electronic screening tool identified significantly higher rates of behavioral health disorders than those that have been previously reported for this patient population.

Electronic risk screening using patient-reported outcome measures offers an efficient approach to improving the identification of behavioral health problems and improving the rates of follow-up care.

 		275
Ainsworth et al [15] Digital interventions on:

Feasibility, adherence, retention, asthma-specific QoL, and asthma control

 		Positive effect

Both the intervention group and the control group improved from baseline to 3 and 12-mo follow-up, with numerically larger improvements in the asthma-related patient-reported outcomes measuring QoL and symptom control (AQLQc and AQCd) at both time points.

Patients in the intervention group who completed 3-mo follow-up measures had mean improvement in asthma-related QoL and in the control group, with the between-group score difference higher in the intervention group, indicating improvements in QoL.

There was no difference in the number of patients who showed minimal clinically important difference improvement at 3 mo (AQLQ>0.5) across groups. The same was true at 12 mo.

By 12 mo, the between-group difference had risen higher in the intervention group.

 		88
Steele Gray et al [45] ePRO mobile app and portal system on:

Enable goal-oriented care delivery and health-related QoL

The secondary outcome: self-management

 		Null

Patients with ePRO combined with usual care demonstrated a nonsignificant decrease in QoL compared with usual care only.

ePRO combined with usual care demonstrated a nonsignificant decrease in patient activation.

No patterns were evident when exploring descriptive trends in outcomes related to ePRO user intensity.

The qualitative data from this study suggest that patients wanted more of a coaching approach with more touch points and interactions to maintain momentum, particularly for patients who started strong but then fizzled out. This finding suggests the need to better calibrate coherence when implementing digital health solutions with diverse user groups over time.

 		44
Harle et al [28] PROe data in an EHRf on:

Providers and patient satisfaction with chronic noncancer pain care

 		Null

During the PRO phase, patients’ mean overall ratings of their visits were not significantly different between groups

Similarly, patients’ mean perceptions that their “doctor did everything he/she could to help you with your pain or discomfort” and confidence “that in the future your pain or discomfort will be well controlled” did not differ significantly between the control and intervention groups in either phase.

The intervention group’s satisfaction increased slightly, but not significantly, by the end of the education phase as well as from the end of the education phase to the PRO phase completion. A similar result was observed for each of the 4 satisfaction subscale measures. Furthermore, after adjusting for baseline satisfaction, there were no differences observed between the intervention and control groups within phases.

The largest control versus intervention difference was a 0.20 higher mean satisfaction with time spent during visits with patients with chronic pain in the intervention group at the end of the PRO phase. However, this difference only approached significance.

 		370
Kroenke et al [29] PROMISg symptom scores to clinicians on:

The PROMIS profile-29 includes scales (sleep, pain, anxiety, depression, and fatigue)

 		Null

There were no differences between feedback and control group patients. Simple feedback of symptom scores to primary care clinicians in the absence of additional systems support or incentives is not superior to usual care in improving symptom outcomes.

Satisfaction did not differ between study groups.

 		256
Lear et al [30]

Internet-based self-management and symptom monitoring program on:

Number of all-cause hospitalizations from the time of randomization to the end of 2 y.

Secondary outcomes: hospital length of stay, QoL, self-management, and social support

 		Null

There were no differences between the 2 groups with respect to changes in QoL. Self-management significantly changed in favor of the internet CDMh intervention in 4 of the 8 domains: skill and technique acquisition, self-monitoring and insight, social integration and support, and emotional well-being.

Social support significantly changed in favor of the internet CDM intervention in 2 of the 5 domains: emotional and informational support and overall support index.

An internet-based self-management program did not result in a significant reduction in hospitalization.

However, fewer participants in the intervention group were admitted to the hospital or experienced the composite outcome of all-cause hospitalization or death. These findings suggest that the internet CDM program has the potential to augment primary care among patients with multiple chronic diseases.

 		229
Miranda et al [31] ePRO tool on:

Estimate the cost of ePRO tool with quality-adjusted life year. Resource use and effectiveness of the ePRO tool

 		Null

Compared with standard care, the ePRO intervention was associated with higher costs. The ePRO tool is not a cost-effective technology for routine assessment.

Compared with standard care, the ePRO intervention was associated with fewer QALYsi (−0.03).

No statistical difference in health-related QoL between ePRO and usual care groups.

The tool would be considered cost-effective if it yielded an improvement of at least 0.03 QALYs.

Long-term and the societal impacts of ePRO were not included in this analysis. Further research is needed to better understand its impact on long-term outcomes and in real-world settings.

 		45
Baron and Duffecy [32] Technology-assisted sleep extension on:

Feasibility and preliminary efficacy of a technology-assisted sleep extension. Predictors of improvement in sleep and blood pressure

 		Positive effect

Technology-assisted sleep extension intervention is feasible and well liked in this population. there was significant improvement in sleep variables in the intervention compared to the self-management group for TSTj.

Intervention adherence, use, and participant feedback: participants in the intervention group wore the Fitbit for 85% to 100% of study days and completed 90% of coaching sessions. All participants reported that they liked the intervention, and ease of the intervention was reported as 4 and 5 by 8 out of 9 participants. One participant rated ease of the intervention as 3.

Change in blood pressure: there were significantly greater changes for 24-h systolic blood pressure and diastolic blood pressure in the intervention group compared with the self-management group (P=.02).

 		16
Bauer et al [46] Mobile health platform supporting collaborative care on:

Feasibility and acceptability of a mobile health platform. Daily surveys of mood and medication and weekly 9-item Patient Health Questionnaire and Generalized Anxiety Disorder 7-item

 		Positive effect

The feasibility and acceptability of the mobile platform is supported by the high early response rate; however, attrition was steep.

 		17
Ramallo-Fariña et al [34] Interventions of knowledge transfer and behavior modification on:

Knowledge transfer and behavior modification

 		Positive effect

The PTIk group is significantly more adherent to the diet recommendations, compared with UCl, after 12 mo of follow-up.

No differences were found in medication adherence, compared with UC.

Compared with UC, both PFIm and CBIn show statistically significant differences at 12 mo for depression and anxiety. These differences disappear at 24 mo because all groups of patients improved.

The diabetes distress score improved significantly compared with the UC group for CBI at 12 mo and for PTI and PFI at 24 mo.

Health-related QoL significantly improved for all intervention groups at 12 mo compared with UC; however, this difference was only sustained for PTI at 18 mo.

Neuropathic symptom scores were significantly lower for the CBI group at 12 mo (P=.02) compared with the UC group (the analysis of nonimputed data led to a nonsignificant result. This difference disappeared at 24 mo).

While average scores were higher than 9 (on a total of 10), in all dimensions, for the group educational sessions, satisfaction with the web platform and SMS text message obtained scores >8.

 		2334
Tamisier et al [35] Telemonitoring intervention on:

Treatment adherence, QoL, functional status in symptomatic patients, health outcomes short form, and the Pichot Fatigue Scale

 		Null

The mean nightly use of CPAPo was similar in the 2 groups.

Using adherence categories, the intention-to-treat analysis showed similar high CPAP adherence in the 2 groups. This was also found in the per-protocol analysis.

Following 6 mo of CPAP treatment, both groups exhibited substantial improvements in daily fatigue and excessive sleepiness, with no significant differences between the groups.

There was an improvement in QoL with CPAP treatment that was significant only in the 12-item short form survey mental component in the TMp group but with no significant differences between groups.

Lipid control, and more specifically total blood cholesterol levels, improved significantly more in the TM group. This improvement was mainly driven by a decrease in low-density lipoprotein cholesterol levels. However, there was no change in BMI.

As expected, there were a much greater number of interventions in the TM group. In patients with obstructed sleep apnea and at least 1 intervention, there was an overall 59% increase in the number of physician interventions and 54% in the number of home care provider’s interventions.

 		206
Trick et al [36] ACASIq system on:

Time burden for patients

 		Positive effect

An ACASI software system can be included in a patient visit and adds minimal time burden.

The burden was greatest for older patients, interviews in Spanish, and for those with less computer exposure. A patient’s self-reported health had minimal impact on response times.

 		1670
Yanicelli et al [37] Home telemonitoring system on:

Self-care treatment adherence and rehospitalization

 		Positive effect

After the 3-mo follow-up period, significant differences were found in the intragroup analysis of the CGr, which highlights a decreased treatment adherence at the end of the study in this group.

Regarding intergroup analysis, there were no significant differences in treatment adherence between groups.

After the 3-mo follow-up period, the mean EHFScBs indicates an improvement in self-care for patients from the IGt and a decrease in self-care for patients from the CG.

Similarly, the intragroup analysis pointed out to significant differences in self-care within both the groups.

There was no significant difference in rehospitalizations between CG and IG.

 		30
Owen-Smith et al [33] Automate PRO data collection on:

Compare study-enhanced PRO completion rates to those in clinical care

 		Positive effect

Adherence to pain-related PRO data collection using our enhanced tiered approach was high.

No demographic or clinical identifiers other than age were associated with differential response by modality.

Using automated modalities is feasible and may facilitate better sustainability for regular PRO administration within health care systems.

 		632

aePRO: electronic patient-reported outcome.

bQoL: quality of life.

cAQLQ: Asthma Quality of Life Questionnaire.

dAQC: acceptable quality level.

ePRO: patient-reported outcome.

fEHR: electronic health record.

gPROMIS: Patient-Reported Outcomes Measurement Information System.

hCDM: chronic disease management.

iQALY: quality-adjusted life year.

jTST: total sleep time.

kPTI: patient-therapist interaction.

lUC: usual care.

mPFI: Physical Function Index.

nCBI: cognitive behavioral intervention.

oCPAP: continuous positive airway pressure.

pTM: telemonitoring.

qACASI: audio computer-assisted self-interview.

rCG: control group.

sEHFScB: European Heart Failure Self-care Behaviour scale.

tIG: intervention group.

 Clinical Outcomes and Quality of Life

A tablet-based electronic screening tool significantly improved the identification of behavioral health problems, offering an efficient approach to behavioral health screening and follow-up health care [27]. Overall, 3 studies showed no improvement on symptoms [29,34,35,43], while another study indicated a positive association [32]. The use of ePROMs in asthma management showed improvements within group in asthma-related quality of life and symptom control but were not statistically significant between groups [15]. The adoption of an ePROMs tool did not demonstrate statistical differences in overall or subscale scores of quality of life, patient activation, or satisfaction with chronic pain health care across both control and intervention groups [29,43,45,47]. Quality of life was used as the dependent variable by 5 studies, 3 showed no improvements [30,35,45], 1 showed improvement [15], and 1 showed decline [31].

 Self-Management and Health Behaviors

Some studies showed significant improvement in self-management [37], skill and technique acquisition, and social integration [30], as well as adherence to diet recommendations and mental health outcomes over time [34]. Two studies showed no improvements in patient activation [45] and self-management [33].

 System-Level Outcomes

Two studies indicated that implementing ePROMs does not reduce rehospitalizations [30,37]. Moreover, compared to usual care, the implementation of ePROMs is associated with a drop in cost-effectiveness because the data collection targets all patients and not specific cases. In addition, differences in patients’ digital literacy may also explain this drop in cost-effectiveness [31]. The study by Staeheli et al [27] showed an improvement in follow-up care.

 Feasibility and Acceptability

Four studies indicated that ePROMs’ implementation is feasible and achieves a good level of acceptability [15,32,33,46], while 2 studies showed no association with satisfaction [28,29]. One study focusing on mental health outcomes also highlighted significant feasibility and acceptability as well as high engagement [46]. However, it also reported a steep attrition in long-term data collection [46]. One study implementing an audio computer-assisted self-interview system showed minimal time burden and high feasibility for patients regardless of age, language, and computer literacy [36].

 Discussion Principal Findings

It is difficult to draw clear conclusions about the effects of ePROM implementation in PHC. However, combining insights from both qualitative and quantitative data on the implementation and effects of ePROMs yields a nuanced understanding of their potential and challenges. A lack of digital literacy and engagement seems to be a key barrier to effectiveness, and in quantitative studies, ePROMs were well received when researchers emphasized feasibility and acceptability and provided training and support. This suggests that improving digital literacy and ensuring user-friendly design and adequate support are crucial for enhancing patient engagement with ePROMs and their effectiveness. Qualitative findings highlighted that the lack of personalization and patient-provider communication were impactful. Quantitatively, interventions that allowed for personalized feedback, goal setting, and self-management support showed positive effects. This indicates that ePROMs that are personalized and facilitate or enhance communication between patients and providers can lead to better health outcomes and patient experiences.

Qualitative data provided insights into perceived benefits, such as increased confidence and motivation and improved person-centered health care, even when quantitative outcomes showed null effects regarding clinical metrics such as health status and quality of life. This suggests a complex relationship between perceived benefits and measurable health outcomes, indicating that ePROMs may impact aspects of health care and patient experience not fully captured by quantitative measures. It also underscores the nuanced effects of digital and behavioral interventions on self-management and behavioral outcomes, highlighting the importance of personalized, interactive approaches and the potential for digital platforms to support but not fully substitute for comprehensive health care strategies aimed at enhancing patient engagement and self-management capabilities [30,33,34,45]. Qualitative insights suggest that despite the enthusiasm for ePROMs and digital interventions, there are concerns about their long-term impact and cost-effectiveness, with quantitative data echoing these concerns. While qualitative data highlight benefits such as increased confidence and motivation and improved patient-centered care, these effects do not always translate into measurable clinical outcomes. This highlights the need for more comprehensive research into the long-term outcomes, cost-effectiveness, and broader societal utility of digital health interventions.

Our review found that most studies were conducted in the United States (9/22, 41%) and Canada (8/22, 36%). The relative emphasis on these regions means that the findings may be influenced by North American health care financing models and policy environments. For instance, in the United States, fragmented insurance coverage and variable reimbursement structures can influence the adoption of digital health tools. In Canada, publicly funded health care and provincial eHealth strategies might support ePROM implementation by offering centralized funding or infrastructure but can also slow adoption. Other high-income countries may have distinct funding mechanisms, regulatory requirements, or national digital health agendas that shape ePROM uptake differently. With most included studies conducted in North America, our results have limited generalizability. Health care systems worldwide vary in digital infrastructure, reimbursement models, and regulatory frameworks, influencing ePROMs adoption. Future research should explore cross-cultural differences and assess implementation in diverse settings to better inform global policy and practice.

The results of this study corroborate those of other systematic reviews. First, systematic reviews carried out in pediatric health care [13], oncology [48,49], or breast cancer treatment [14] also present divergent results about patient satisfaction, quality of health care, health outcomes, patient management, and patient health behavior. The authors of these studies conclude that differences in the context of health care and the quality of ePROM implementation make it difficult to generalize the results of the studies that have been identified. Ishaque et al [48] also associates this divergence to the low statistical power of most of the studies reviewed. Second, the authors also conclude that ePROM implementations have the potential to improve patient health, provided they consider the barriers and facilitators specific to each health care setting [14,48,50].

Some of our results also differ from those of systematic reviews that have looked at other health care contexts. Indeed, systematic reviews indicate a positive, clear, and well-supported relationship between the implementation of ePROMs and health benefits. For example, in pediatrics, the integration of ePROMs is associated with an increase in health-related quality of life and patient satisfaction [13], while in oncology, the detection of unidentified problems and the monitoring of treatment response are improved by the implementation of ePROMs [49]. This difference in results from our study can probably be explained by the greater heterogeneity of populations and treatments in PHC, pointing at the significant challenge to widespread ePROMs implementation and uptake in this setting.

 Limitations

Our study has several limitations. The heterogeneity of study designs and outcome measures precluded meta-analysis, necessitating a narrative synthesis. The quantitative studies included in our sample assessed a diverse range of health indexes using scales that were not based on a common metric, making it impossible to extract standardized results for quantification. Consequently, we interpreted their findings using the vote-counting method [51]. However, the vote-counting method has been criticized [51], particularly because it does not account for effect sizes, limiting comparability and resulting in lower precision than meta-analysis. Our qualitative data analysis was limited to a descriptive approach. To accurately capture the net effect of ePROM implementation, future studies should aim to minimize the influence of barriers while maximizing the impact of facilitators. To advance this understanding, we plan to conduct a meta-synthesis of the identified qualitative studies, offering a comprehensive analysis of barriers, facilitators, and their interrelationships.

In terms of public decision-making, we can question the real size of the social benefits of implementing ePROMs. Indeed, only 1 of the studies in our sample carried out a cost-benefit analysis [31], and none of them did so in an ecological manner. No study has compared the benefits of implementing ePROMs to those of another intervention with the same level of financial investment. Future research should conduct cost-benefit analyses in a real-world context to identify which sector would yield the greatest overall benefit from financial investment.

Success in implementing ePROMs in PHC appears to hinge on addressing digital literacy, ensuring personalization and meaningful patient-provider interactions, carefully integrating technology into clinical workflows, and conducting thorough research on their long-term impacts and cost-effectiveness. ePROM implementation research could conduct large‐scale, multisite randomized trials or pragmatic trials to compare ePROM implementation strategies; investigate long-term outcomes and sustainability (eg, cost‐utility analyses and ecological cost‐benefit comparisons); focus on diverse patient populations, especially underserved communities, to address health equity and digital literacy gaps; and use standardized outcome measures or validated ePROM instruments to facilitate cross‐study comparisons. By focusing on these areas, future efforts could better assess the benefits of digital health technologies for patients, providers, and health care systems.

PRISMA checklist.

Search strategy.

Digital tools used for electronic patient-reported outcome measures’ collection and implementation in electronic medical records.

 Abbreviations ePROM

electronic patient-reported outcome measure

 MMAT

Mixed Methods Appraisal Tool

 PHC

primary health care

 PICOS

Population, Intervention, Comparison, Outcomes, and Setting

 PRISMA

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

 RE-AIM

reach, effectiveness, adoption, implementation, and maintenance

The authors would like to acknowledge the insight and impact of their deceased colleague Pierre Pluye, who helped plan the mixed methods methodology, the funding proposal, and the protocol of this study, as well as inspire them as researchers. This research was funded by the Canadian Institute for Health Research.

None declared.

 Primary health care World Health Organization 2024-01-01 https://www.who.int/health-topics/primary-health-care#tab=tab_1 Declaration of Alma-Ata World Health Organization 2019 10 8 2024-01-01 https://www.who.int/publications/i/item/WHO-EURO-1978-3938-43697-61471 Why Canada’s health system needs (a lot more) team-based care Canadian Medical Association 2024-01-01 https://tinyurl.com/5xz7j5s5 Access to primary care Office of Disease Prevention and Health Promotion, Office of the Assistant Secretary for Health, Office of the Secretary, U.S. Department of Health and Human Services 2024-01-01 https://odphp.health.gov/healthypeople/priority-areas/social-determinants-health/literature-summaries/access-primary-care About chronic diseases Centers for Disease Control and Prevention 2024 10 4 2024-01-01 https://www.cdc.gov/chronicdisease/about/index.htm Betancourt MT Roberts KC Bennett TL Driscoll ER Jayaraman G Pelletier L Monitoring chronic diseases in Canada: the Chronic Disease Indicator Framework Chronic Dis Inj Can 2014 06 34 Supplement 1 1 30 10.24095/hpcdp.34.s1.01 Evaluation of the Strategy for Patient-Oriented Research (SPOR) Canadian Institutes of Health Research 2023 6 2024-01-01 https://cihr-irsc.gc.ca/e/53635.html Nguyen H Butow P Dhillon H Sundaresan P A review of the barriers to using Patient-Reported Outcomes (PROs) and Patient-Reported Outcome Measures (PROMs) in routine cancer care J Med Radiat Sci 2021 06 19 68 2 186 95 10.1002/jmrs.421 32815314 PMC8168064 Roux-Lévy PH Poitras ME Supper W Attisso E Dugas M LeBlanc A Bergeron F Sasseville M Systèmes de mesures auto-raportées pour guider la prise de décisions poliWques SPOR Evidence Alliance 2023 11 18 2024-01-01 https://sporevidencealliance.ca/wp-content/uploads/2023/11/SPOREA_CSBE-Decision-Making-Report_Final.pdf Glenngård AH Anell A Process measures or patient reported experience measures (PREMs) for comparing performance across providers? A study of measures related to access and continuity in Swedish primary care Prim Health Care Res Dev 2017 9 15 19 01 23 32 10.1017/s1463423617000457 Gwaltney CJ Shields AL Shiffman S Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review Value Health 2008 03 11 2 322 33 10.1111/j.1524-4733.2007.00231.x 18380645 S1098-3015(10)60526-8 Meirte J Hellemans N Anthonissen M Denteneer L Maertens K Moortgat P Van Daele U Benefits and disadvantages of electronic patient-reported outcome measures: systematic review JMIR Perioper Med 2020 04 03 3 1 e15588 10.2196/15588 33393920 v3i1e15588 PMC7709853 Bele S Chugh A Mohamed B Teela L Haverman L Santana MJ Patient-reported outcome measures in routine pediatric clinical care: a systematic review Front Pediatr 2020 7 28 8 364 10.3389/fped.2020.00364 32850521 PMC7399166 van Egdom LS Oemrawsingh A Verweij LM Lingsma HF Koppert LB Verhoef C Klazinga NS Hazelzet JA Implementing patient-reported outcome measures in clinical breast cancer care: a systematic review Value Health 2019 10 22 10 1197 226 10.1016/j.jval.2019.04.1927 31563263 S1098-3015(19)32152-7 Ainsworth B Greenwell K Stuart B Raftery J Mair F Bruton A Yardley L Thomas M Feasibility trial of a digital self-management intervention 'My Breathing Matters' to improve asthma-related quality of life for UK primary care patients with asthma BMJ Open 2019 11 12 9 11 e032465 10.1136/bmjopen-2019-032465 31722952 bmjopen-2019-032465 PMC6858238 Higgins JP Green S Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series 2008 London, UK The Cochrane Collaboration Page MJ McKenzie JE Bossuyt PM Boutron I Hoffmann TC Mulrow CD Shamseer L Tetzlaff JM Moher D Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement J Clin Epidemiol 2021 06 134 103 12 10.1016/j.jclinepi.2021.02.003 33577987 S0895-4356(21)00040-8 Chien PF Khan KS Siassakos D Registration of systematic reviews: PROSPERO BJOG 2012 07 17 119 8 903 5 10.1111/j.1471-0528.2011.03242.x 22703418 Sasseville M Supper W Gartner JB Layani G Amil S Sheffield P Gagnon MP Hudon C Lambert S Attisso E Bureau Lagarde V Breton M Poitras ME Pluye P Roux-Levy PH Plaisimond J Bergeron F Ashcroft R Wong S Groulx A Beaudet N Paquette JS D'Anjou N Langlois S LeBlanc A Clinical integration of digital patient-reported outcome measures in primary health care for chronic disease management: protocol for a systematic review JMIR Res Protoc 2023 08 18 12 e48155 10.2196/48155 37594780 v12i1e48155 PMC10474503 Hupe M EndNote X9 J Electron Resour Med Libr 2019 11 26 16 3-4 117 9 10.1080/15424065.2019.1691963 Page MJ McKenzie JE Bossuyt PM Boutron I Hoffmann TC Mulrow CD Shamseer L Tetzlaff JM Akl EA Brennan SE Chou R Glanville J Grimshaw JM Hróbjartsson A Lalu MM Li T Loder EW Mayo-Wilson E McDonald S McGuinness LA Stewart LA Thomas J Tricco AC Welch VA Whiting P Moher D The PRISMA 2020 statement: an updated guideline for reporting systematic reviews BMJ 2021 03 29 372 n71 10.1136/bmj.n71 33782057 PMC8005924 Hong QN Pluye P Fàbregues S Bartlett G Boardman F Cargo M Dagenais P Gagnon MP Griffiths F Nicolau B O'Cathain A Rousseau MC Vedel I Improving the content validity of the mixed methods appraisal tool: a modified e-Delphi study J Clin Epidemiol 2019 07 111 49 59.e1 10.1016/j.jclinepi.2019.03.008 30905698 S0895-4356(18)30082-9 Glasgow RE Dickinson P Fisher L Christiansen S Toobert DJ Bender BG Dickinson LM Jortberg B Estabrooks PA Use of RE-AIM to develop a multi-media facilitation tool for the patient-centered medical home Implement Sci 2011 10 21 6 1 118 10.1186/1748-5908-6-118 22017791 1748-5908-6-118 PMC3229439 Glasgow RE Vogt TM Boles SM Evaluating the public health impact of health promotion interventions: the RE-AIM framework Am J Public Health 1999 09 89 9 1322 7 10.2105/ajph.89.9.1322 10474547 PMC1508772 Hong QN Pluye P Bujold M Wassef M Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence Syst Rev 2017 03 23 6 1 61 10.1186/s13643-017-0454-2 28335799 10.1186/s13643-017-0454-2 PMC5364694 Laux G Kuehlein T Rosemann T Szecsenyi J Co- and multimorbidity patterns in primary care based on episodes of care: results from the German CONTENT project BMC Health Serv Res 2008 01 18 8 1 14 10.1186/1472-6963-8-14 18205916 1472-6963-8-14 PMC2244601 Staeheli M Aseltine RH Jr Schilling E Anderson D Gould B Using mHealth technologies to improve the identification of behavioral health problems in urban primary care settings SAGE Open Med 2017 06 08 5 2050312117712656 10.1177/2050312117712656 28634539 10.1177_2050312117712656 PMC5467966 Harle CA Marlow NM Schmidt SO Shuster JJ Listhaus A Fillingim RB Hurley RW The effect of EHR-integrated patient-reported outcomes on satisfaction with chronic pain care Am J Manag Care 2016 12 01 22 12 e403 8 27982672 86886 PMC5460672 Kroenke K Talib TL Stump TE Kean J Haggstrom DA DeChant P Lake KR Stout M Monahan PO Incorporating promis symptom measures into primary care practice-a randomized clinical trial J Gen Intern Med 2018 08 5 33 8 1245 52 10.1007/s11606-018-4391-0 29623512 10.1007/s11606-018-4391-0 PMC6082211 Lear SA Norena M Banner D Whitehurst DG Gill S Burns J Kandola DK Johnston S Horvat D Vincent K Levin A Kaan A Van Spall HG Singer J Assessment of an interactive digital health-based self-management program to reduce hospitalizations among patients with multiple chronic diseases: a randomized clinical trial JAMA Netw Open 2021 12 01 4 12 e2140591 10.1001/jamanetworkopen.2021.40591 34962560 2787457 Miranda RN Bhuiya AR Thraya Z Hancock-Howard R Chan BC Steele Gray C Wodchis WP Thavorn K An electronic patient-reported outcomes tool for older adults with complex chronic conditions: cost-utility analysis JMIR Aging 2022 04 20 5 2 e35075 10.2196/35075 35442194 v5i2e35075 PMC9069297 Baron K Duffecy J Technology assisted behavior intervention to extend sleep among adults with short sleep duration and prehypertension/stage 1 hypertension: a randomized pilot feasibility study Sleep Med 2019 12 64 S26 10.1016/j.sleep.2019.11.073 Owen-Smith A Mayhew M Leo MC Varga A Benes L Bonifay A DeBar L Automating collection of pain-related patient-reported outcomes to enhance clinical care and research J Gen Intern Med 2018 05 9 33 Suppl 1 31 7 10.1007/s11606-018-4326-9 29633139 10.1007/s11606-018-4326-9 PMC5902345 Ramallo-Fariña Y Rivero-Santana A García-Pérez L García-Bello MA Wägner AM Gonzalez-Pacheco H Rodríguez-Rodríguez L Kaiser-Girardot S Monzón-Monzón G Guerra-Marrero C Daranas-Aguilar C Roldán-Ruano M Carmona M Serrano-Aguilar PG Patient-reported outcome measures for knowledge transfer and behaviour modification interventions in type 2 diabetes-the INDICA study: a multiarm cluster randomised controlled trial BMJ Open 2021 12 15 11 12 e050804 10.1136/bmjopen-2021-050804 34911711 bmjopen-2021-050804 PMC8679133 Tamisier R Treptow E Joyeux-Faure M Levy P Sapene M Benmerad M Bailly S Grillet Y Stach B Muir JF Pegliasco H Pépin JL Impact of a multimodal telemonitoring intervention on CPAP adherence in symptomatic OSA and low cardiovascular risk: a randomized controlled trial Chest 2020 11 158 5 2136 45 10.1016/j.chest.2020.05.613 32640196 S0012-3692(20)31865-1 Trick WE Deamant C Smith J Garcia D Angulo F Implementation of an audio computer-assisted self-interview (ACASI) system in a general medicine clinic Appl Clin Inform 2017 12 19 06 01 148 62 10.4338/aci-2014-09-ra-0073 Yanicelli LM Goy CB González VD Palacios GN Martínez EC Herrera MC Non-invasive home telemonitoring system for heart failure patients: a randomized clinical trial J Telemed Telecare 2020 01 23 27 9 553 61 10.1177/1357633x19899261 Bezerra Giordan L Ronto R Chau J Chow C Laranjo L Use of mobile apps in heart failure self-management: qualitative study exploring the patient and primary care clinician perspective JMIR Cardio 2022 04 20 6 1 e33992 10.2196/33992 35442205 v6i1e33992 PMC9069281 Steele Gray C Gill A Khan AI Hans PK Kuluski K Cott C The electronic patient reported outcome tool: testing usability and feasibility of a mobile app and portal to support care for patients with complex chronic disease and disability in primary care settings JMIR Mhealth Uhealth 2016 06 02 4 2 e58 10.2196/mhealth.5331 27256035 v4i2e58 PMC4911509 Hans PK Gray CS Gill A Tiessen J The provider perspective: investigating the effect of the Electronic Patient-Reported Outcome (ePRO) mobile application and portal on primary care provider workflow Prim Health Care Res Dev 2017 9 13 19 02 151 64 10.1017/s1463423617000573 Irfan Khan A Gill A Cott C Hans PK Steele Gray C mHealth tools for the self-management of patients with multimorbidity in primary care settings: pilot study to explore user experience JMIR Mhealth Uhealth 2018 08 28 6 8 e171 10.2196/mhealth.8593 30154073 v6i8e171 PMC6134226 Schoenthaler A Cruz J Payano L Rosado M Labbe K Johnson C Gonzalez J Patxot M Patel S Leven E Mann D Investigation of a mobile health texting tool for embedding patient-reported data into diabetes management (i-Matter): development and usability study JMIR Form Res 2020 08 31 4 8 e18554 10.2196/18554 32865505 v4i8e18554 PMC7490676 Steele Gray C Gravesande J Hans PK Nie JX Sharpe S Loganathan M Lyons R Cott C Using exploratory trials to identify relevant contexts and mechanisms in complex electronic health interventions: evaluating the electronic patient-reported outcome tool JMIR Form Res 2019 02 27 3 1 e11950 10.2196/11950 30810532 v3i1e11950 PMC6414821 Ahmed S Zidarov D Eilayyan O Visca R Prospective application of implementation science theories and frameworks to inform use of PROMs in routine clinical care within an integrated pain network Qual Life Res 2021 11 02 30 11 3035 47 10.1007/s11136-020-02600-8 32876812 10.1007/s11136-020-02600-8 PMC8528775 Steele Gray C Chau E Tahsin F Harvey S Loganathan M McKinstry B Mercer SW Nie JX Palen TE Ramsay T Thavorn K Upshur R Wodchis WP Assessing the implementation and effectiveness of the electronic patient-reported outcome tool for older adults with complex care needs: mixed methods study J Med Internet Res 2021 12 02 23 12 e29071 10.2196/29071 34860675 v23i12e29071 PMC8726765 Bauer AM Iles-Shih M Ghomi RH Rue T Grover T Kincler N Miller M Katon WJ Acceptability of mHealth augmentation of collaborative care: a mixed methods pilot study Gen Hosp Psychiatry 2018 03 51 22 9 10.1016/j.genhosppsych.2017.11.010 29272712 S0163-8343(17)30287-6 PMC6512981 Harle CA Listhaus A Covarrubias CM Schmidt SO Mackey S Carek PJ Fillingim RB Hurley RW Overcoming barriers to implementing patient-reported outcomes in an electronic health record: a case report J Am Med Inform Assoc 2016 01 23 1 74 9 10.1093/jamia/ocv085 26159464 ocv085 PMC5009936 Ishaque S Karnon J Chen G Nair R Salter AB A systematic review of randomised controlled trials evaluating the use of patient-reported outcome measures (PROMs) Qual Life Res 2019 03 3 28 3 567 92 10.1007/s11136-018-2016-z 30284183 10.1007/s11136-018-2016-z Chen J Ou L Hollis SJ A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting BMC Health Serv Res 2013 06 11 13 1 211 10.1186/1472-6963-13-211 23758898 1472-6963-13-211 PMC3700832 Glenwright BG Simmich J Cottrell M O'Leary SP Sullivan C Pole JD Russell T Facilitators and barriers to implementing electronic patient-reported outcome and experience measures in a health care setting: a systematic review J Patient Rep Outcomes 2023 02 14 7 1 13 10.1186/s41687-023-00554-2 36786914 10.1186/s41687-023-00554-2 PMC9928985 Rothman KJ Greenland S Lash TL Modern Epidemiology 2008 Philadelphia, PA Lippincott Williams & Wilkins