This paper is in the following e-collection/theme issue:

Viewpoints and Perspectives (550) Patient-Reported Outcome Measures (PROMs) (314) Clinical Communication, Electronic Consultation and Telehealth (654) Participatory Medicine & E-Patients (819) Digital Health, Telehealth and e-Innovation in Clinical Settings (457)

Published on in Vol 27 (2025)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/60777, first published .
Improving the Implementation of Patient-Reported Outcome Measure in Clinical Practice: Tackling Current Challenges With Innovative Digital Communication Technologies

Improving the Implementation of Patient-Reported Outcome Measure in Clinical Practice: Tackling Current Challenges With Innovative Digital Communication Technologies

Improving the Implementation of Patient-Reported Outcome Measure in Clinical Practice: Tackling Current Challenges With Innovative Digital Communication Technologies

Authors of this article:

Kelly M de Ligt1, 2 Author Orcid Image ;   Saar Hommes3 Author Orcid Image ;   Ruben D Vromans3 Author Orcid Image ;   Eva Boomstra4 Author Orcid Image ;   Lonneke V van de Poll1, 2, 5 Author Orcid Image ;   Emiel J Krahmer3 Author Orcid Image
Article Authors Cited by Tweetations (1) Metrics

    Viewpoint

    1Department of Psychosocial Research and Epidemiology, Netherlands Cancer Institute, Amsterdam, Netherlands

    2Department of Medical and Clinical Psychology, Center of Research on Psychological and Somatic Disorders, Tilburg University, Tilburg, Netherlands

    3Department Communication and Cognition, Tilburg School of Humanities and Digital Sciences, Tilburg University, Tilburg, Netherlands

    4Department of Research and Development, Netherlands Comprehensive Cancer Organisation, Utrecht, Netherlands

    Corresponding Author:

    Kelly M de Ligt, PhD

    Department of Psychosocial Research and Epidemiology

    Netherlands Cancer Institute

    PO Box 90203

    Amsterdam, 1006 BE

    Netherlands

    Phone: 31 2051269111

    Email: k.d.ligt@nki.nl


    Implementation of patient-reported outcome measures (PROMs) in clinical practice is challenging. We believe effective communication is key to realizing the clinical benefits of PROMs. Communication processes for PROMs in clinical practice typically involve (1) health care professionals (HCPs) inviting patients to complete PROMs, (2) patients completing PROMs, (3) HCPs and patients interpreting the resulting patient-reported outcomes (PROs), and (4) HCPs and patients using PROs for health management. Yet, communication around PROMs remains underexplored. Importantly, patients differ in their skills, knowledge, preferences, and motivations for completing PROMs, as well as in their ability and willingness to interpret and apply PROs in managing their health. Despite this, current communication practices often fail to account for these differences. This paper highlights the importance of personalized communication to make PROMs accessible to diverse populations. Personalizing communication manually is highly labor-intensive, but several digital technologies can offer a feasible solution to accommodate various patients. Despite their potential, these technologies have not yet been applied to PROMs. We explore how existing principles and tools, such as automatic data-to-text generation (including multimodal outputs like text combined with data visualizations) and conversational agents, can enable personalized communication of PROMs in practice.

    J Med Internet Res 2025;27:e60777

    doi:10.2196/60777

    Keywords

    patient reported outcome measures; quality of life; health communication; delivery of health care; digital sciences; clinical practice: patient reported outcomes; patient reported outcome; digital communication; communication; health management; digital technologies 


    Patient-reported outcome measures (PROMs) are instruments that can capture patients’ health status directly through digital administration [1-3]. Patients vary in their skills, knowledge, preferences, and motivations to complete PROMs, as well as their ability to interpret and use the resulting patient-reported outcomes (PROs) for health management. However, PROMs and PROs are not designed to accommodate these differences, often excluding certain populations [4]. This includes older adults, nonnative speakers, individuals with poor health, those lacking social support, people in less privileged socioeconomic positions, or those with low health literacy—defined as “the degree to which individuals can obtain, process, and understand the basic health information and services they need to make appropriate health decisions” [4-6].

    Communication is crucial for achieving the clinical benefits of PROMs, for instance when health care professionals (HCPs) discuss PROs with patients. However, the communication processes involved in using PROMs in clinical practice are underexplored [7-9]. We therefore argue that improving communication processes can enhance PROMs implementation. Specifically, personalized communication (information that is customized to the individual) is essential to address the diverse information needs, preferences, and capacities of a broad population [10]. However, personalizing information manually is labor-intensive and costly, which may explain its limited application to PROMs.

    To address this, we propose leveraging existing digital technologies to streamline personalized communication for PROMs. Tools such as data-to-text generation, multimodal communication, and conversational agents can offer innovative solutions to improve communication and, in turn, support the broader implementation of PROMs in clinical practice. This paper begins by summarizing communication processes for PROMs in clinical practice and identifying their shortcomings. These shortcomings highlight the limitations of a one-size-fits-all approach, which fails to meet the needs of a diverse patient population. We propose solutions rooted in personalized communication and suggest several digital technologies to support the implementation of these strategies in clinical practice.


    From many available frameworks, we use the framework of Lasswell [11] to outline key aspects of communication for PROMs in clinical practice: who communicates what, in what form, to whom, and to what effect (Figure 1). We distinguish 4 communication processes, that are PROMs invitation, PROMs completion, PRO presentation, and PRO-based health management.

    ‎
    Figure 1. Communication processes for PROMs in clinical practice. HCPs: health care professionals; HRQoL: health-related quality of life; PROMs: patient-reported outcome measures; PROs: patient-reported outcomes.

    First, HCPs or administrative staff (“who”) invite patients (“to whom”) verbally or by email (“in what form”), including the goal and practical guidance on how to complete PROMs (“what”) [11,12]. This aims to engage patients to complete PROMs (“to what effect”).

    Second, PROMs (“what”) are administered through online portals (“in what form”) for patients (“to whom”) to complete (“to what effect”).

    Third, HCPs (“who”) should consistently discuss PROs (“what”) with patients (“to whom”) during consultations (“in what form”) [12,13], providing insights into their health and health-related quality of life (HRQoL; “to what effect”) [12]. Alternatively, patients can occasionally access their PRO reports directly via electronic portals (“in what form”), bypassing HCPs.

    Fourth, HCPs (“who”) can translate the PROs into actionable health management information (“what”) to aid patients’ (“to whom”) health management (“to what effect”). Similarly, patients may independently access this information (“what”) through electronic portals (“in what form”), bypassing HCPs.


    The shortcomings of each communication process are detailed below, along with solutions based on personalized communication and digital technologies to support implementation. Table 1 summarizes the identified shortcomings, proposed solutions, and relevant digital technologies.

    Table 1. Improving communication processes around PROMsa: identified shortcomings, proposed solutions, and relevant digital technologies.
    Identified shortcomingsProposed solutionsDigital technologies to support implementation
    1: PROMs invitation: tailoring invitations by data-to-text to accommodate more patients

    		 Not all patients understand the purpose of completing PROMs, when PROsb will be discussed, and how they benefit, resulting in varying levels of motivation to complete PROMs.Written invitations could be tailored to accommodate different reading and health literacy levels to better serve patients with varying literacy and health literacy levels.“Data-to-text” can tailor PROMs invitations to better serve patients with varying literacy and health literacy levels in an automated, data-driven way.
    2: PROMs completion: multimodal communication principles to support PROMs completion

    		Not all patients are able to complete PROMs, as they have diverse skills, knowledge, and preferences. Yet, PROMs are presented in a single, standardized format.Presenting PROMs in other formats other than text could better serve patients with lower (health) literacy.Multimodal communication, such as a video of an interviewer reading the questions, with written text that changes color in synch with the audiovisual playback.

    		Not all patients are able to complete PROMs, as they have diverse skills, knowledge, and preferences. Yet, PROMs are presented in a single, standardized format.Allowing patients to verbally complete PROMs could better serve patients with lower literacy.Conversational agents integrated into PROMs administration systems would allow patients to complete PROMs verbally.
    3: PRO presentation: data-to-text to convert numeric PROs into comprehensible texts

    		 Not all patients understand PROs because of the way these are presented, and HCPsc frequently omits to explain PROs to patients.Presenting PROs in a more patient-friendly way to patients.Data-to-text applications can convert numeric PROs into written texts or narratives.
    4: PRO-based health management: from PROs to health management by conversational agents

    		HCPs fail to guide patients to use PROs for health management, or patients perceive health management information as not relevant as their information needs differ between patients.Both HCPs and patients require guidance on how to use PROs for health management.Conversational agents can break down information into smaller bits, highlight relevant PROs, and provide self-management advice. By tailoring PROs and self-management actions to individual patients, information becomes more relevant.

    aPROMs: patient-reported outcome measures.

    bPROs: patient-reported outcomes.

    cHCPs: health care professionals.

    PROMs Invitation: Tailoring Invitations by Data-To-Text to Accommodate More Patients

    Not all patients understand the purpose of completing PROMs when PROs will be discussed, and how they will benefit, leading to varying levels of motivation to complete PROMs. Some patients may perceive PROMs as tools for research or to train HCPs [12-15]. Written invitations could be tailored to different reading and health literacy levels to better serve patients with varying literacy and health literacy levels, ensuring information about the goal and use of PROMs are clearly communicated, potentially improving completion rates.

    “Data-to-text” is a digital technology with the potential for tailoring PROMs invitations to diverse audiences in a data-driven manner. A form of natural language generation (NLG), it uses insights from computational linguistics and artificial intelligence (AI) [16]. This technology automates the conversion of data into coherent natural language, typically achieved through a series of computational steps [17]. A standard data-to-text pipeline processes a patient’s data, using a sequence of algorithms to determine the content and structure of the output text. By incorporating self-reported or clinically obtained information about patients' education, literacy, and health literacy levels, the system could generate texts at various readability levels to accommodate a range of reading proficiencies [18].

    PROMs Completion: Multimodal Communication Principles to Support PROMs Completion

    Patients have diverse skills, knowledge, and preferences for completing PROMs [4] (Table 1). While digital PROMs can be offered through multiple completion methods [19,20], and flexible administration is recommended [21], often only a single method is provided. As a result, not all patients can complete PROMs. Those facing barriers are particularly older, non-White, lower-educated patients, and those with low health literacy [4,22]. Reducing these barriers could improve completion rates in these groups.

    Data-to-text could tailor PROMs to patients’ reading levels, but textual PROMs may still be too lengthy, information-dense, or intimidating for some patients. According to dual coding theory [23] and the cognitive theory of multimedia learning [24], multimodal communication (especially combining text with visuals) may help alleviate these issues [18]. These theories suggest that our working memory processes different types of information through separate channels, each with limited capacity. Using multiple channels, such as visual and verbal information, enhances information transfer. For example, integrating images with text can significantly improve comprehension, as long as the visuals and text are presented together and complement each other [24-27].

    Examples of multimodal PROMs already exist [20,28-31], such as the multimedia program of Thumboo et al [28], which delivers PROMs via a touchscreen device. Each question is paired with visual and auditory stimuli – a video of an interviewer reading the question, with accompanying text that changes color in sync with the audio playback. The playback speed can be adjusted to accommodate patients with different reading levels. This approach could be extended to PROM invitations, questions, and PRO-related texts, improving accessibility and comprehension.

    Furthermore, conversational agents, which emulate human conversation through text or speech [32], can be integrated into PROMs administration systems, allowing patients to complete PROMs verbally using familiar devices such as computers, smartphones, or tablets [31]. We believe this approach could be especially effective when combined with multimodal or interactive voice response systems that read questions aloud [20,31,33], making PROMs completion more accessible, especially for less literate patients. In addition, conversational agents can overcome language barriers by supporting multiple languages, promoting inclusivity for nonnative speakers [34]. Examples of this have been demonstrated by Mlakar et al [30] and Fenza et al [35], though formal evaluations of their relevance and efficiency are still needed.

    PRO Presentation: Data-To-Text to Convert Numeric PROs Into Comprehensible Texts

    Although patients want to reflect on their health and HRQoL before discussions with their HCP [13,14], many are unable to do so. That is, digital PROMs typically grant HCPs but not patients’ access to PROs [7]. Despite this, HCPs often neglect to discuss PROs [8], and patients are generally hesitant to initiate these conversations [13]. When patient do have access to their PROs, these are typically presented in numerical or graphical formats. While patients have expressed a need for personalized numerical information [36,37] and can interpret basic line or bar graphs [19], such formats often lack context and evaluability (ie, how good or bad the scores are) [38], leading to misinterpretation [19,39]. PRO comprehension is influenced by patient’s health literacy [40] and statistical literacy or numeracy skills [41]. However, despite patients’ varying levels for both, PROs are typically presented in a single, fixed format [19]. We believe that improving how PROs are presented will enhance their relevance and value for patients, providing better and more equitable access to the potential benefits of PROs in patient care.

    Data-to-text applications could convert numeric PRO scores into written texts, combining numerical information with verbal descriptors to enhance understanding and improve information transfer [42], as studies suggest [12,18,19,43]. We believe both patients and HCPs could benefit from this approach. When discussing individualized outcome data, it is recommended to combine numerical or visual information with verbal descriptors to better align with patients’ needs and capacities [44]. While some patients and HCPs prefer verbal descriptors over “cold” numbers or graphs for discussing outcomes and HRQoL data [36,45], relying solely on words can lead to varied and inaccurate perceptions. Despite its potential, the practical use of NLG for converting PROs into coherent texts is not yet common in clinical settings. The Dutch “Data2Text” project represents an initial exploration into this promising field [46].

    Patient narratives, or stories about other patients’ experiences, can help patients understand the meaning of their own PROs [37,42]. When patients recognize PROMs as reflections of their health experiences, HCPs can engage in discussions about how PROs can enhance HRQoL and care [14]. As a form of data-to-text, personalized narratives could be generated based on patients’ PRO scores. This approach was explored in previous research by our group, where it was found to be feasible for clinical practice. Importantly, we discovered that narratives can provide emotional support, complementing numeric data to facilitate discussions about HRQoL during clinical consultations [37]. With further development, a decision algorithm could select relevant stories from a collection of patient narratives, using a patient’s PROs as input.

    PROs Presentation—Based Health Management: From PROs to Health Management by Conversational Agents

    In one-third of studies reviewed by Anatchkova et al [3], HCPs failed to guide patients on PRO-based health management. In addition, patients typically use health information in decision-making only when the information is relevant to them, and information needs vary between patients (Table 1) [42]. Improving patients’ access to personalized health information could potentially enhance their care and support better health-related outcomes.

    Both patients and many HCPs need guidance on how PROs contribute to care, and how to interpret, discuss, and act on PROs [8,13,14]. Conversational agents can break down information into smaller, manageable pieces, acting as intermediaries by highlighting relevant PROs and providing self-management advice [42,47]. Examples include conversational agents developed for asthma self-management [48] and to promote healthy eating behaviors [49]. For patient dashboards, cognitive load can be reduced by highlighting key information and enhancing evaluability [42]. When PROs are presented numerically or graphically, emphasizing the clinical significance of scores aids interpretation. This can be achieved through color-coding (eg, red for alarming and green for non-alarming scores), using exclamation marks or red circles for critical scores, and adding threshold lines to indicate scores relative to clinical thresholds [19]. We believe this approach could facilitate PRO interpretation and clinical decision-making, accelerating the use of PROs in health management. In addition, for patients, simply receiving health information is not enough — it must be interesting and relevant to them, which varies between patients [42]. Based on patients’ predefined goals and preferences, conversational agents can tailor PRO scores and self-management actions to meet their specific information needs.


    With increasing pressure on health care, digital strategies for patient self-management are expected to play a larger role [50,51]. Paradoxically, those who would benefit the most from enhanced self-management support (such as vulnerable populations) are often the ones facing barriers to participation. For instance, lower health literacy is associated with worse health outcomes and healthcare service utilization [52], while lower digital health literacy correlates with lower overall survival rates in cancer patients [53]. While PROMs have the potential to improve symptom management, HRQoL, patient-HCP communication, and patient satisfaction [7,12,13,54,55], vulnerable patients may currently lack access to these benefits. By implementing personalized communication strategies and leveraging digital technologies, we believe PROMs can become more accessible and easier to complete, thereby increasing completion rates. Furthermore, making PROs easier to interpret can improve their usability and value for patients, ensuring that PROMs can more effectively support patient care, particularly for those currently left out.

    Despite promising advances, the use of the digital applications described here is not without risks. Data-to-text algorithms, particularly those based on data-driven approaches, present certain challenges. Traditionally, data-to-text systems relied on hand-crafted rules, which were difficult to scale. However, recent advances in machine learning, driven by increased computing power, have streamlined this process, enabling scalability and efficiency [18,56]. Large language models, such as exemplified by ChatGPT, are arguably the culmination of this approach. While these models generate fluent text, they are not always accurate and may contain harmful biases. Large language models rely on massive volumes of human-authored texts, typically sourced from the internet, raising concerns about the exclusion of underrepresented populations (eg, those with a small presence on the internet) and the inclusion of biased or toxic language [57,58]. Therefore, rule-based NLG might be preferable for converting PROs into text, offering more control and reliability. Similarly, while advances in machine learning have enhanced conversational agents with increased dialogue complexity, they introduce risks in natural language understanding, response generation, and patient interpretation, necessitating rigorous monitoring and validation [59]. The safety of conversational agents for patients remains inadequately evaluated [32]. We emphasize that these digital applications require further exploration and testing, despite their clear promise.

    In addition to personalized communication, patient-centered communication could improve the use of PROMs in practice [21]. Currently, patients face issues with wording, length of questionnaires, and formatting of PROMs, which can hinder their understanding of questions and answer choices [15,34,39]. Over half of the commonly used cancer-related PROMs to not follow best-practices for plain language use [60]. To improve completion rates, existing PROMs should be revised to use simple, concrete language, avoiding jargon and negations, and eliminate redundant or distracting information [15,61]. Instructions should also be simplified [61-63] and structured with clear orientation statements (eg, “First I will explain the goals of PROMs, then I will show you what the questions are about”) to aid understanding [61,62]. Resources like the “health literacy universal precautions toolkit” can help assess readability and convert medical terms into plain language [63].

    To reduce patient burden, PROM should be relevant for individuals [21]. Computer adaptive testing can ensure follow-up questions are tailored to each patient, rather than asking the same set for everyone. The formatting of PROMs should prioritize clarity, with large fonts, clear headings, and ample white space [15,61]. Patient-preferred modifications include removing or altering items, using visuals, simplifying language, and adjusting layouts for better comprehension [29]. Using user-centered design can help meet the diverse needs of patients, considering factors such as clinical conditions, culture, languages, and literacy levels [4]. This approach should also explicitly involve individuals with low literacy skills or learning disabilities, as they are often excluded from PROMs development [64]. By doing so, we can ensure that PROMs are accessible to a wide range of patients, including those with disabilities [20].

    Training HCPs about the goals, tasks, and responsibilities related to PROMs is essential for motivating their engagement with PROMs [65,66]. In general, HCPs should discuss or, at a minimum, briefly mention PROs with patients, even when no new clinical information arises from the data [13]. HCP engagement plays a significant role in patients’ motivation to complete PROMs [67]; failing to follow up on PROMs may negatively affect patients’ HRQoL [68]. Further research is needed to assist HCPs in interpreting PROs and converting them into actionable steps for health management [3,13]. That said, some patients may avoid engaging with detailed PRO data [36], highlighting the need for tailoring information to meet patients’ individual preferences for how they receive and process information. Although digital technologies can support communication around PROMs in clinical practice, they may not be able, now or ever, to replace the HCP in addressing patients’ HRQoL.


    Implementing the proposed technologies in clinical practice might present some practical hurdles. For instance, many of the identified applications have not been seamlessly integrated into electronic health records (EHRs) [47,69]. Future health care systems will require EHRs that can safely incorporate digital applications. However, models could be trained without sharing data from EHRs, using techniques like federated learning [70]. Furthermore, trust, which is crucial for patient adherence [71], is challenged by the perceived “black box” nature of AI models. Therefore, it is essential to have human oversight of AI outputs and to maintain transparency about data sources and methods [59]. Both HCPs and patients should become more AI-literate, and have a basic understanding of the capabilities and limitations of AI models [70]. To ensure that readers can assess the credibility of generated texts, it is essential to explain that texts come from a computer model, and not a fellow human [57].

    The successful implementation of PROMs in clinical practice involves complex interactions between care processes, technological development, and human behavior [66]. Often, PROMs implementation is led by HCPs or researchers, potentially resulting in a lack of recognition of these complex interactions. This paper was written by a multidisciplinary team with expertise in PROMs, implementation science, health communication, and digital technologies. We have provided a comprehensive overview of key communication processes around PROMs. Future work could deepen this assessment by studying the effects of context and noise in communication, which are aspects currently missing from the application of the Lasswell model [11]. With ongoing advancements in AI, which may help mitigate the risks associated with the proposed digital applications, we envision a future where personalized communication strategies and digital tools can overcome current implementation challenges, ensuring that PROMs are applied inclusively to benefit all patients.

    Acknowledgments

    This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. ChatGPT was used to check the manuscript for correct grammar and language use.

    Data Availability

    Data sharing is not applicable to this article as no data sets were generated or analyzed during this study.

    Authors' Contributions

    All authors have been involved in the conceptualization of the manuscript. Author KMdL has written the original draft, which has been reviewed and edited by authors SH, RDV, EB, LVvdP, and EJK. All authors have approved the submitted version of the manuscript.

    Conflicts of Interest

    None declared.

    1. U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research;, U.S. Department of Health and Human Services FDA Center for Biologics Evaluation and Research, U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health Qual Life Outcomes. 2006;4:79. [FREE Full text] [CrossRef] [Medline]
    2. Giordano FA, Welzel G, Siefert V, Jahnke L, Ganslandt T, Wenz F, et al. Digital follow-Up and the perspective of patient-centered care in oncology: what's the PROblem? Oncology. 2020;98(6):379-385. [CrossRef] [Medline]
    3. Anatchkova M, Donelson SM, Skalicky AM, McHorney CA, Jagun D, Whiteley J. Exploring the implementation of patient-reported outcome measures in cancer care: need for more real-world evidence results in the peer reviewed literature. J Patient Rep Outcomes. 2018;2(1):64. [FREE Full text] [CrossRef] [Medline]
    4. Calvert MJ, Cruz Rivera S, Retzer A, Hughes SE, Campbell L, Molony-Oates B, et al. Patient reported outcome assessment must be inclusive and equitable. Nat Med. 2022;28(6):1120-1124. [FREE Full text] [CrossRef] [Medline]
    5. Gebert P, Schindel D, Frick J, Schenk L, Grittner U. Characteristics and patient-reported outcomes associated with dropout in severely affected oncological patients: an exploratory study. BMC Med Res Methodol. 2021;21(1):77. [FREE Full text] [CrossRef] [Medline]
    6. Institute of Medicine Committee on Health Literacy. Nielsen-Bohlman L, Panzer AM, Kindig DA, editors. Health Literacy: A Prescription to End Confusion. Washington (DC). National Academies Press (US); 2004.
    7. Gibbons C, Porter I, Gonçalves-Bradley DC, Stoilov S, Ricci-Cabello I, Tsangaris E, et al. Routine provision of feedback from patient-reported outcome measurements to healthcare providers and patients in clinical practice. Cochrane Database Syst Rev. 2021;10(10):CD011589. [FREE Full text] [CrossRef] [Medline]
    8. Greenhalgh J, Gooding K, Gibbons E, Dalkin S, Wright J, Valderas J, et al. How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis. J Patient Rep Outcomes. 2018;2:42. [FREE Full text] [CrossRef] [Medline]
    9. Greenhalgh J, Long AF, Flynn R. The use of patient reported outcome measures in routine clinical practice: lack of impact or lack of theory? Soc Sci Med. 2005;60(4):833-843. [CrossRef] [Medline]
    10. Noar SM, Grant Harrington N, Van Stee SK, Shemanski Aldrich R. Tailored health communication to change lifestyle behaviors. American Journal of Lifestyle Medicine. 2010;5(2):112-122. [CrossRef]
    11. Lasswell HD. Bryson L, editor. The structurefunction of communication in society. New York, NY. Harper; 1948.
    12. Eriksen J, Bygholm A, Bertelsen P. The association between patient-reported outcomes (PROs) and patient participation in chronic care: a scoping review. Patient Educ Couns. 2022;105(7):1852-1864. [FREE Full text] [CrossRef] [Medline]
    13. van der Willik EM, Milders J, Bart JAJ, Bos WJW, van Ittersum FJ, Ten Dam MAGJ, et al. Discussing results of patient-reported outcome measures (PROMs) between patients and healthcare professionals in routine dialysis care: a qualitative study. BMJ Open. 2022;12(11):e067044. [FREE Full text] [CrossRef] [Medline]
    14. Lai CH, Shapiro LM, Amanatullah DF, Chou LB, Gardner MJ, Hu SS, et al. A framework to make PROMs relevant to patients: qualitative study of communication preferences of PROMs. Qual Life Res. 2022;31(4):1093-1103. [CrossRef] [Medline]
    15. Jahagirdar D, Kroll T, Ritchie K, Wyke S. Using patient reported outcome measures in health services: a qualitative study on including people with low literacy skills and learning disabilities. BMC Health Serv Res. 2012;12:431. [FREE Full text] [CrossRef] [Medline]
    16. Reiter E, Dale R. Building applied natural language generation systems. Nat. Lang. Eng. 1997;3(1):57-87. [CrossRef]
    17. Hommes S, van der Lee D, Clouth F, Vermunt J, Verbeek X, Krahmer E. A Personalized Data-to-Text Support Tool for Cancer Patients. 2019. Presented at: Proceedings of the 12th International Conference on Natural Language Generation, Tokyo, Japan Association for Computational Linguistics. ?52; 2019 Oct 29:443; Tokyo, Japan. [CrossRef]
    18. Pauws S, Gatt A, Krahmer E, Reiter E. Making effective use of healthcare data using data-to-text technology. In: Consoli S, Reforgiato Recupero D, Petkovi M, editors. Data Science for Healthcare: Methodologies and Applications. Cham. Springer International Publishing; 2019.
    19. Albers EAC, Fraterman I, Walraven I, Wilthagen E, Schagen SB, van der Ploeg IM, et al. Visualization formats of patient-reported outcome measures in clinical practice: a systematic review about preferences and interpretation accuracy. J Patient Rep Outcomes. 2022;6(1):18. [FREE Full text] [CrossRef] [Medline]
    20. Harniss M, Amtmann D, Cook D, Johnson K. Considerations for developing interfaces for collecting patient-reported outcomes that allow the inclusion of individuals with disabilities. Med Care. 2007;45(5 Suppl 1):S48-S54. [FREE Full text] [CrossRef] [Medline]
    21. Aiyegbusi OL, Cruz Rivera S, Roydhouse J, Kamudoni P, Alder Y, Anderson N, et al. Recommendations to address respondent burden associated with patient-reported outcome assessment. Nat Med. 2024;30(3):650-659. [CrossRef] [Medline]
    22. Pugh SL, Rodgers JP, Yeager KA, Chen RC, Movsas B, Bonanni R, et al. Characteristics of participation in patient-reported outcomes and electronic data capture components of NRG oncology clinical trials. Int J Radiat Oncol Biol Phys. 2020;108(4):950-959. [FREE Full text] [CrossRef] [Medline]
    23. Paivio A. Dual coding theory: retrospect and current status. Canadian Journal of Psychology / Revue canadienne de psychologie. 1991;45(3):255-287. [CrossRef]
    24. Mayer RE. Cognitive theory of multimedia learning. In: Mayer RE, editor. The Cambridge Handbook of Multimedia Learning. Cambridge. Cambridge University Press; 2005.
    25. Schnotz W. An integrated model of textpicture comprehension. In: Mayer RE, editor. The Cambridge Handbook of Multimedia Learning. Cambridge. Cambridge University Press; 2005.
    26. Chandler P, Sweller J. Cognitive load theory and the format of instruction. Cognition & Instruction. 1991;8(4):293-332.
    27. Pusic MW, Ching K, Yin HS, Kessler D. Seven practical principles for improving patient educationvidence-based ideas from cognition science. Paediatr Child Health. 2014;19:119-122. [CrossRef]
    28. Thumboo J, Wee HL, Cheung YB, Machin D, Luo N, Fong KY. Development of a smiling touchscreen multimedia program for HRQoL assessment in subjects with varying levels of literacy. Value Health. 2006;9(5):312-319. [FREE Full text] [CrossRef] [Medline]
    29. Milte R, Jemere D, Lay K, Hutchinson C, Thomas J, Murray J, et al. A scoping review of the use of visual tools and adapted easy-read approaches in quality-of-Life instruments for adults. Qual Life Res. 2023;32(12):3291-3308. [FREE Full text] [CrossRef] [Medline]
    30. Mlakar I, Šafran V, Hari D, Rojc M, Alankuş G, Pérez LR, et al. Multilingual conversational systems to drive the collection of patient-reported outcomes and integration into clinical workflows. Symmetry. 2021;13(7):1187. [CrossRef]
    31. Golden AH, Gabriel MH, Russo J, Price M, Ruhmel S, Nilsson A, et al. Let’s talk about it: an exploration of the comparative use of three different digital platforms to gather patient-reported outcome measures. J Patient Rep Outcomes. 2023;7(1):1187. [CrossRef]
    32. Laranjo L, Dunn AG, Tong HL, Kocaballi AB, Chen J, Bashir R, et al. Conversational agents in healthcare: a systematic review. J Am Med Inform Assoc. 2018;25(9):1248-1258. [FREE Full text] [CrossRef] [Medline]
    33. Hahn EA, Cella D, Dobrez D, Shiomoto G, Marcus E, Taylor SG, et al. The talking touchscreen: a new approach to outcomes assessment in low literacy. Psychooncology. 2004;13(2):86-95. [CrossRef] [Medline]
    34. Kilbridge KL, Fraser G, Krahn M, Nelson EM, Conaway M, Bashore R, et al. Lack of comprehension of common prostate cancer terms in an underserved population. J Clin Oncol. 2009;27(12):2015-2021. [FREE Full text] [CrossRef] [Medline]
    35. Fenza G, Orciuoli F, Peduto A, Postiglione A. Healthcare Conversational Agents: Chatbot for Improving Patient-Reported Outcomes. Cham. Springer International Publishing; 2023.
    36. Vromans RD, Hommes S, Clouth FJ, Lo-Fo-Wong DNN, Verbeek XAAM, van de Poll-Franse L, et al. Need for numbers: assessing cancer survivors' needs for personalized and generic statistical information. BMC Med Inform Decis Mak. 2022;22(1):260. [FREE Full text] [CrossRef] [Medline]
    37. Boomstra E, Hommes S, Vromans RD, van der Burg S, Schrijver AM, Wouters MWJM, et al. "Numbers call for action, personalized narratives provide support and recognition": a qualitative assessment of cancer patients' perspectives on patient-reported outcome measures (PROMs) feedback with narratives. J Cancer Surviv. 2024. [CrossRef] [Medline]
    38. Zikmund-Fisher BJ. Helping people know whether measurements have good or bad implications: increasing the evaluability of health and science data communications. Policy Insights from the Behavioral and Brain Sciences. 2019;6(1):29-37.
    39. Grossman LV, Feiner SK, Mitchell EG, Masterson Creber RM. Leveraging patient-reported outcomes using data visualization. Appl Clin Inform. 2018;9(3):565-575. [FREE Full text] [CrossRef] [Medline]
    40. Turchioe MR, Myers A, Isaac S, Baik D, Grossman LV, Ancker JS, et al. A systematic review of patient-facing visualizations of personal health data. Appl Clin Inform. 2019;10(4):751-770. [FREE Full text] [CrossRef] [Medline]
    41. Gigerenzer G, Gaissmaier W, Kurz-Milcke E, Schwartz LM, Woloshin S. Helping doctors and patients make sense of health statistics. Psychol Sci Public Interest. 2007;8(2):53-96. [CrossRef] [Medline]
    42. Hibbard JH, Peters E. Supporting informed consumer health care decisions: data presentation approaches that facilitate the use of information in choice. Annu Rev Public Health. 2003;24:413-433. [CrossRef] [Medline]
    43. van der Horst DEM, van Uden-Kraan CF, Parent E, Bart JAJ, Waverijn G, Verberk-Jonkers IJAM, et al. Optimizing the use of patients' individual outcome information - development and usability tests of a chronic kidney disease dashboard. Int J Med Inform. 2022;166:104838. [FREE Full text] [CrossRef] [Medline]
    44. Vromans RD, van Eenbergen MC, Geleijnse G, Pauws S, van de Poll-Franse LV, Krahmer EJ. Exploring cancer survivor needs and preferences for communicating personalized cancer statistics from registry data: qualitative multimethod study. JMIR Cancer. 2021;7(4):e25659. [FREE Full text] [CrossRef] [Medline]
    45. Vromans RD, Tillier CN, Pauws SC, van der Poel HG, van de Poll-Franse LV, Krahmer EJ. Communication, perception, and use of personalized side-effect risks in prostate cancer treatment-decision making: an observational and interview study. Patient Educ Couns. 2022;105(8):2731-2739. [FREE Full text] [CrossRef] [Medline]
    46. Krahmer EJ, Clouth FJ, Hommes S, Vromans RD, Pauws SC, Vermunt JK. Helping cancer patients to choose the best treatment: towards automated data-driven and personalized information presentation of cancer treatment options. In Commit2Data. Open Access Series in Informatics (OASIcs). 2024;124:1-20. [CrossRef]
    47. Lau-Min KS, Marini J, Shah NK, Pucci D, Blauch AN, Cambareri C, et al. Pilot study of a mobile phone chatbot for medication adherence and toxicity management among patients with GI cancers on capecitabine. JCO Oncol Pract. 2024;20(4):483-490. [CrossRef]
    48. Kadariya D, Venkataramanan R, Yip HY, Kalra M, Thirunarayanan K, Sheth A. kBot: knowledge-enabled personalized chatbot for asthma self-management. 2019. Presented at: IEEE International Conference on Smart Computing (SMARTCOMP); 2019 June 2-15; Washington, DC, USA. [CrossRef]
    49. Fadhil A, Gabrielli S. Addressing challenges in promoting healthy lifestyles: the al-chatbot approach. 2017. Presented at: Proceedings of the 11th EAI International Conference on Pervasive Computing Technologies for Healthcare; Barcelona, Spain: Association for Computing Machinery; 2017 May 23 - 26,; USA. [CrossRef]
    50. Chan RJ, Howell D, Lustberg MB, Mustian K, Koczwara B, Ng CC, et al. Advances and future directions in the use of mobile health in supportive cancer care: proceedings of the 2019 MASCC Annual Meeting symposium. Support Care Cancer. 2020;28(9):4059-4067. [CrossRef] [Medline]
    51. Aminabee S. The future of healthcare Patient-centric care: digital innovations, trends, and predictions. In: Garcia MB, de Almeida RPP, editors. Emerging Technologies for Health Literacy Medical Practice. Hershey PA USA. IGI Global; 2024.
    52. Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155(2):97-107. [CrossRef] [Medline]
    53. Heudel PE, Delrieu L, Dumas E, Crochet H, Hodroj K, Charrier I, et al. Impact of limited E-Health literacy on the overall survival of patients with cancer. JCO Clin Cancer Inform. 2022;6:e2100174. [FREE Full text] [CrossRef] [Medline]
    54. Campbell R, Ju A, King MT, Rutherford C. Perceived benefits and limitations of using patient-reported outcome measures in clinical practice with individual patients: a systematic review of qualitative studies. Qual Life Res. 2022;31(6):1597-1620. [CrossRef] [Medline]
    55. Graupner C, Kimman ML, Mul S, Slok AHM, Claessens D, Kleijnen J, et al. Patient outcomes, patient experiences and process indicators associated with the routine use of patient-reported outcome measures (PROMs) in cancer care: a systematic review. Support Care Cancer. 2021;29(2):573-593. [FREE Full text] [CrossRef] [Medline]
    56. Gatt A, Krahmer E. Survey of the state of the art in natural language generation: core tasks, applications and evaluation. jair. 2018;61:65-170. [CrossRef]
    57. Bender E, Gebru T, McMillan-Major A, Shmitchell S. On the dangers of stochastic parrots: can language models be too big. 2021. Presented at: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency; Virtual Event, Canada:; 2021 March 1; Canada. [CrossRef]
    58. van Dis EAM, Bollen J, Zuidema W, van Rooij R, Bockting C. ChatGPT: five priorities for research. Nature. 2023;614(7947):224-226. [CrossRef] [Medline]
    59. Smiley C, Schilder F, Plachouras V, Leidner J. Say the right thing rightthics issues in natural language generation systems. 2017. Presented at: Proceedings of the First ACL Workshop on Ethics in Natural Language Processing. ?8; 2017 April 4; Valencia Spain. [CrossRef]
    60. Papadakos JK, Charow RC, Papadakos CJ, Moody LJ, Giuliani ME. Evaluating cancer patient-reported outcome measures: readability and implications for clinical use. Cancer. 2019;125(8):1350-1356. [FREE Full text] [CrossRef] [Medline]
    61. Wilson EAH, Wolf MS. Working memory and the design of health materials: a cognitive factors perspective. Patient Educ Couns. 2009;74(3):318-322. [CrossRef] [Medline]
    62. Weiss BD. Literacy and Patient Safety: Help Patients Understand - Manual for Clinicians. 2018;2nd edition. 2018. URL: https:/​/psnet.​ahrq.gov/​issue/​health-literacy-and-patient-safety-help-patients-understand-manual-clinicians-2nd-ed [accessed 2015-09-02]
    63. DeWalt DA, Broucksou KA, Hawk VH, Brach C, Hink A, Rudd R, et al. Health Literacy Universal Precautions Toolkit. In: (Prepared by North Carolina Network Consortium, The Cecil G. Sheps Center for Health Services Research, The University of North Carolina at Chapel Hill, under Contract No. Rockville, MD. Agency for Healthcare Research and Quality; 2010.
    64. Jahagirdar D, Kroll T, Ritchie K, Wyke S. Patient-reported outcome measures for chronic obstructive pulmonary disease : the exclusion of people with low literacy skills and learning disabilities. Patient. 2013;6(1):11-21. [FREE Full text] [CrossRef] [Medline]
    65. Boyce MB, Browne JP, Greenhalgh J. The experiences of professionals with using information from patient-reported outcome measures to improve the quality of healthcare: a systematic review of qualitative research. BMJ Qual Saf. 2014;23(6):508-518. [FREE Full text] [CrossRef] [Medline]
    66. Austin E, LeRouge C, Hartzler AL, Chung AE, Segal C, Lavallee DC. Opportunities and challenges to advance the use of electronic patient-reported outcomes in clinical care: a report from AMIA workshop proceedings. JAMIA Open. 2019;2(4):407-410. [FREE Full text] [CrossRef] [Medline]
    67. Unni E, Coles T, Lavallee DC, Freel J, Roberts N, Absolom K. Patient adherence to patient-reported outcome measure (PROM) completion in clinical care: current understanding and future recommendations. Qual Life Res. 2024;33(1):281-290. [FREE Full text] [CrossRef] [Medline]
    68. Velikova G, Sheppard S, Campbell L, Smith A, Awad N, Selby P, et al. Randomized trial of quality-of-life measurement in oncology practice: Do oncologists need to know? Journal of Clinical Oncology. 2008;26(15):9586. [CrossRef] [Medline]
    69. Giordanengo A, Årsand E, Woldaregay AZ, Bradway M, Grottland A, Hartvigsen G, et al. Design and prestudy assessment of a dashboard for presenting self-collected health data of patients with diabetes to clinicians: iterative approach and qualitative case study. JMIR Diabetes. 2019;4(3):e14002. [FREE Full text] [CrossRef] [Medline]
    70. Meskó B. The impact of multimodal large language models on health care's future. J Med Internet Res. 2023;25:e52865. [FREE Full text] [CrossRef] [Medline]
    71. Martin LR, Williams SL, Haskard KB, Dimatteo MR. The challenge of patient adherence. Ther Clin Risk Manag. 2005;1(3):189-199. [FREE Full text] [Medline]

    ‎
    AI: artificial intelligence
    EHR: electronic health record
    HCP: health care professional
    HRQoL: health-related quality of life
    NLG: natural language generation
    PRO: patient-reported outcome
    PROM: patient-reported outcome measure

    Edited by N Cahill; submitted 21.05.24; peer-reviewed by BZ Stern, PD Franklin; comments to author 18.09.24; revised version received 07.11.24; accepted 18.12.24; published 05.02.25.

    Copyright

    ©Kelly M de Ligt, Saar Hommes, Ruben D Vromans, Eva Boomstra, Lonneke V van de Poll, Emiel J Krahmer. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 05.02.2025.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.