Effective health communication is recognized as a public health priority [1]. Health communication aims to improve health by ensuring effective understanding and application of health information. Central to this is health literacy, defined as the degree to which individuals have the capacity to obtain and understand health information needed to make appropriate health decisions [2]. Patients with higher health literacy are more likely to engage in health-promoting behaviors, utilize health care services, and effectively manage chronic diseases [3]. However, the use of complex medical language poses a significant barrier to patient understanding. Health care professionals are encouraged to use plain language tailored to patients’ comprehension levels, which can be challenging, especially in time-sensitive clinical care settings [4,5]. Patient education materials (PEMs) play a central role in supporting patient-physician interactions by providing clear and accessible information on health conditions, treatments, and health promotion [6]. Personalized PEMs have been shown to improve patient care through shared decision-making, enhanced patient satisfaction, and better physical and psychosocial well-being [7]. To maximize accessibility of PEMs, leading organizations such as the National Institutes of Health and the American Medical Association (AMA) recommend writing PEMs at or below a sixth-grade reading level (RGL) [8,9]. However, numerous studies have demonstrated that a significant portion of existing PEMs fail to meet this benchmark, often being written at a level too complex for many patients to understand. An analysis of PEMs showed average readability scores ranging from 8th to 15th RGL across various medical fields [10-16]. This readability concern has not improved between 2001 and 2022. These findings underscore that simplified versions of PEMs are necessary, and health care professionals are encouraged to provide easy-to-read PEMs to patients [17,18].

Artificial intelligence (AI) offers promising opportunities to enhance effective health communication. In particular, large language models (LLMs) have emerged as transformative tools in natural language processing (NLP), enabling diverse applications such as answering patient questions; summarizing, translating, or simplifying medical texts; supporting clinical paperwork; and providing individualized medical guidance [19-21]. Importantly, LLMs have the potential to enhance the accessibility of medical knowledge by making complex medical language more comprehensible to laypeople, thereby enabling patients to better understand their health conditions [22,23]. Text simplification as an NLP task has advanced significantly in recent years, especially driven by developments in LLMs since 2019. Whereas earlier technologies relied primarily on rule-based systems or machine learning models, LLMs have enabled a paradigm shift in NLP, making these capabilities more widely accessible to health care professionals and health researchers. In addition, LLM capabilities have rapidly evolved, demonstrating increasingly sophisticated language understanding and generation abilities [24,25].

Despite promising results, important challenges remain. These include the risk of factual errors (hallucinations), critical omissions of information, and the unintended loss of meaning between original and simplified text versions. Furthermore—and most critically—the difficulty of verifying understandability persists, as strong performance on standard quality metrics does not guarantee that simplified texts are actually understandable to laypeople [26]. Moreover, the rapid evolution of AI language processing technologies makes it challenging for researchers and health care providers to maintain an up-to-date overview of available tools and supporting evidence.

This scoping review aimed to identify and map existing evidence on the use of automated language processing technologies for the simplification of PEMs into layperson-friendly language. For this review, layperson-friendly language is defined as text characterized by an accessible reading level, simple sentence structures, and explanation or avoidance of medical jargon. This scoping review is guided by the following research questions:

This scoping review followed the methodological framework of the Joanna Briggs Institute (JBI) [27] and is reported according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) [28] (Multimedia Appendix 1) and the PRISMA-S (PRISMA Statement for Reporting Literature Searches in Systematic Reviews) [29] (Multimedia Appendix 2) guidelines. The review methodology was registered in the Open Science Framework [30].

The preregistered protocol specified the inclusion of studies on automatic text simplification of any complex medical or health-related text. As Figure 1 shows, full-text screening revealed a large and highly heterogeneous body of literature, encompassing different document types such as radiology reports, electronic health records, discharge letters, PEMs, informed consent forms, and scientific papers. This heterogeneity presents challenges for meaningful synthesis and comparison across fundamentally different health information materials with varying purposes, audiences, and complexity levels.

Additionally, this study was framed as part of a larger project, the A+CHIS project [31], which aims to develop a system that provides users with a selection of diverse health documents and preliminary PEMs tailored to their individual information needs and cognitive prerequisites. To present a more focused and coherent synthesis of evidence that would yield insights for patient-facing health communication systems such as A+CHIS, a post hoc decision was made to refine the scope to studies concerning PEM simplification specifically. This narrowed focus aligns with the primary goal of improving health information accessibility for laypeople and ensures methodological consistency in evaluating simplification approaches.

This post hoc decision was made after the initial full-text screening but before full-text data extraction. All other methods remained unchanged from the preregistered protocol.

A comprehensive literature search was conducted in May 2025 across 5 bibliographic databases: MEDLINE, Embase, CINAHL, PsycINFO, and IEEE Xplore. The MEDLINE and Embase strategies were run simultaneously as a multifile search in Ovid, and results were deduplicated using the Ovid deduplication tool. The search strategy was based on database-specific controlled vocabulary and free-text terms. To identify relevant search terms, the MeSH Browser, a keyword analysis tool [32], and a synonym identification tool [33] were used. The search was limited to the period from 2019 to 2025, as this period marks significant advances in automated language processing technologies, particularly with the emergence of LLMs that have substantially enhanced automatic text simplification capabilities. No search filter was applied. The search block for “automated language processing technologies” was based on 2 existing search strategies [34,35]. Detailed search strategies for the databases are provided in Multimedia Appendix 3. The MEDLINE search strategy was peer reviewed by another research team member (KJ).

Reference lists of included studies were manually screened to identify additional studies. We also searched gray literature in July 2025 using the search string “patient education material” AND (“large language model” OR “ChatGPT”) AND “simplifying” in Google. The first 5 pages of each search result were screened. We did not contact the authors of included studies for clarification, as the available data were sufficient to conduct the scoping review. No other methods were used to locate additional studies. Eligible studies were imported into EndNote 21.5 (Clarivate Plc) to identify and remove duplicates.

Following JBI guidelines [27], 2 authors (CK, DW, CL, or NB) independently screened all titles and abstracts against the inclusion criteria. A pilot stage involving 25 titles and abstracts was conducted to ensure consistent application of the criteria, with conflicts resolved through discussion. Full texts of potentially eligible studies were subsequently assessed by 2 independent reviewers (CK, DW, CL, or NB) using the same eligibility criteria. Disagreements were resolved through discussion or by involving a third reviewer (CK, DW, CL, or NB).

A standardized data extraction form was developed and piloted on 3 eligible studies to assess its clarity and completeness. Following the pilot test and discussion within the research team, 1 additional item (the language of simplified PEMs) was added to the final form.

The following data from the included studies were extracted: bibliographic details (authors, year of publication, and country), technology details (name and version of LLM and prompts used), source text (medical field of PEMs, number of analyzed materials, and language), outcomes (text quality indicators and measurement methods), and key findings (main results related to linguistic quality and content fidelity). As recommended by the JBI methodology for scoping reviews [27], 1 reviewer (CK) extracted the data, which was verified by a second reviewer (DW).

In line with JBI guidelines [27], critical appraisal of eligible studies was not required.

Following the JBI scoping review methodology [27], we conducted a descriptive synthesis of the extracted data. We did not perform an analytical synthesis of outcomes but instead mapped and summarized findings descriptively. The extracted data were collated iteratively and organized using a framework-based approach, categorizing findings into the 2 predefined outcome domains: linguistic quality (encompassing linguistic comprehensibility and linguistic correctness) and content fidelity (encompassing factual correctness and factual completeness). Within each domain, descriptive qualitative content analysis was conducted by the first author (CK) to summarize the types of automated language processing technologies evaluated, the measurement methods applied, and the reported direction of effect (eg, improvements in readability scores or identified factual inaccuracies). Results were presented descriptively through narrative summaries, tables, and figures to address the review questions. Frequency counts were used where appropriate to quantify the occurrence of specific technologies or outcome measures. All descriptive analyses were conducted using Microsoft Excel.

The systematic database search identified 7218 references. Following deduplication, 5456 titles and abstracts were screened, and 129 full texts were subsequently assessed for eligibility. This process resulted in the inclusion of 24 [36-59] studies specifically addressing PEMs. In addition, 2 studies [60,61] identified from relevant reviews were included. Two further studies [62,63] were identified through reference list screening of the included studies, and a supplementary Google search yielded 3 additional relevant studies [64-66]. In total, this scoping review included 31 studies focusing on PEMs. Figure 1 shows the study selection process in a PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram.

Studies excluded at the full-text screening stage and their primary reasons for exclusion are provided in Multimedia Appendix 4.

The main characteristics of all included studies are presented in Table 1. All included studies investigated LLMs exclusively, evaluating 8 different models. OpenAI’s GPT series emerged as the most extensively studied models: GPT-4.0 (n=10) [36-43,62,64], GPT-3.5 (n=8) [44-49,60,61], and GPT-3.0 (n=1) [50]. Other evaluated LLMs included Google’s Gemini (n=4) [51,52,65,66] and Bard (n=5) [53-56,63], Anthropic’s Claude (n=2) [57,65], Microsoft’s Copilot (n=1) [66], and Meta’s Llama (n=1) [57], with 12 studies conducting comparative analyses across multiple models [51-59,63,65,66]. As illustrated in Multimedia Appendix 5, all included studies were published very recently, with 11 in 2025 [36-40,44,51,57,64-66], 19 in 2024 [41-43,45-48,50,52-56,58-63], and 1 in 2023 [49]. Although the literature search covered the period from 2019 onward, no relevant studies were identified before 2023, highlighting the emerging nature of this field.

The geographical distribution showed a strong US dominance (n=27) [36-47,49,51-57,59-65], with additional contributions from the United Kingdom [50], Australia [48], the Netherlands [66], and a US-Thailand collaboration [58]. This geographic focus is reflected in the languages studied, with 30 studies examining English PEMs [36-39,41-66], 2 evaluating both English and Spanish PEMs [37,47], and 1 focusing solely on Spanish PEMs [40].

Sample sizes varied considerably, ranging from 7 to 340 PEMs per study, including full texts, text excerpts [45,48], and frequently asked question sections [58].

Figure 2 illustrates the distribution of medical fields addressed in the PEMs in the included studies. Surgery was the most frequently investigated field (n=14), encompassing orthopedics, neurosurgery, plastic surgery, oral and maxillofacial surgery, craniofacial surgery, and otolaryngology. Ophthalmology was the second most common field (n=7), covering topics such as glaucoma, cataract, uveitis, and dry eye disease. Additional fields included internal medicine (n=4; heart disease, aortic stenosis, cancer), neurology (n=3; general neurology, stroke, idiopathic intracranial hypertension), and diagnostic radiology (n=2). Single studies addressed emergency medicine (burns first aid) [50], obstetrics and gynecology (reproductive genetics) [66], and urology (renal transplantation) [58]. No studies were identified in the following medical fields: allergy and immunology, anesthesiology, dermatology, dentistry, family medicine, medical genetics, nuclear medicine, pathology, pediatrics, physical medicine and rehabilitation, preventive medicine, psychiatry, or radiation oncology.

To evaluate the effect of prompt engineering, 1 study [59] compared the outputs of GPT-3.5 and GPT-4.0 using 2 different prompts. The first was a simple prompt with a general instruction to simplify the text while maintaining its structure. The second, more detailed prompt provided explicit constraints, referencing specific readability scales and including examples to guide the model’s output [59]. All prompts used in the included studies are presented in Multimedia Appendix 6.

A comprehensive overview of the reported outcomes across all included studies is provided in Multimedia Appendix 7.

As highlighted in Figure 5, linguistic comprehensibility was predominantly assessed using automated methods, as the manual assessment of objective indicators such as readability, sentence length, or word count is inefficient. By contrast, verifying factual correctness and completeness, as well as evaluating the understandability of simplified PEMs, inherently requires human judgment and cannot be fully automated. While content fidelity of simplified PEMs in the included studies was assessed by experts, this occurred less frequently overall. Crucially, the analysis reveals 2 major validation gaps: first, no study assessed the linguistic correctness of the simplified PEMs. Second, and most notably, no study involved laypeople in assessing the actual understandability of the simplified PEMs.

A total of 6 studies evaluated the understandability and clarity of simplified PEMs through human expert assessments, including research team members or health care professionals, using validated tools such as the Patient Education Materials Assessment Tool [51,54,55,65] and subjective assessments of whether the PEM was understandable for an average patient or successfully improved accessibility without losing key information [52,58]. However, only 4 studies directly examined the relationship between readability improvement, content accuracy, and overall expert-rated understandability of simplified PEMs [54,55,58,65] (Table 3).

The studies confirmed that GPT-3.5, GPT-4.0, Bard, Gemini 1.5, and Claude 3.5 consistently improved readability scores across standard metrics, except for GPT-3.5 using FKGL in 1 analysis [54]. Content accuracy was generally high but varied by LLM and medical field. GPT-3.5, GPT-4.0, and Bard achieved 100% content accuracy across multiple conditions. However, Gemini 1.5 demonstrated 10% inaccuracy for heart disease content while maintaining 100% accuracy for cancer and stroke. Claude 3.5 showed the opposite pattern: 100% accuracy for heart disease but 5% inaccuracy for cancer and stroke. Understandability proved most challenging. GPT-4.0 emerged as the most capable, successfully generating readable, accurate, and understandable materials for childhood glaucoma [55] and kidney donation [62]; however, it failed for idiopathic intracranial hypertension [45]. Other models demonstrated greater limitations: GPT-3.5 and Bard generated readable and accurate outputs that experts deemed not understandable across multiple evaluations. Disease-specific performance varied substantially: GPT-4.0 generated understandable PEMs for cancer but not for cardiovascular conditions, whereas Claude 3.5 achieved understandability only for heart disease content [65].

This scoping review synthesized evidence on automated language processing technologies currently used to simplify PEMs into layperson-friendly language. A total of 31 studies met the inclusion criteria and examined LLMs for automatic text simplification. Notably, despite a comprehensive literature search, no studies specifically examined AI-supported writing assistants (eg, DeepL Write) or other AI-supported tools designed for automated text simplification. This scoping review has 3 key findings.

First, LLMs consistently improve readability metrics, with GPT-4.0 demonstrating the most reliable performance in comparison with other LLMs. However, achieving predefined RGLs remains challenging across all LLMs, particularly for GPT models at the lowest target level (fifth grade). Notably, GPT-4.0 does not consistently outperform GPT-3.5; rather, performance varies by target RGL. This indicates that newer models are not automatically better at achieving specific RGLs than older ones. Although Gemini and Claude show higher success rates, limited analyses preclude definitive conclusions. At the current stage of technological development, LLMs struggle to consistently achieve the recommended RGL for optimal lay comprehension, particularly the sixth-grade level recommended by the AMA. However, based on the available data, it remains unclear whether the failure to consistently achieve target RGLs stems from inherent model limitations or from heterogeneity in prompting strategies across the included studies.

Second, content fidelity assessments showed considerable variability. Although content similarity scores were generally high, content accuracy ranged from 48% to 100% across studies. This level of inaccuracy is concerning in medical contexts, where even minor errors can compromise patient safety.

Third, and most critically, this scoping review reveals a fundamental validation gap: no study has evaluated linguistic correctness (eg, grammar or typographical errors), and no study has assessed whether laypeople actually understand the simplified PEMs. The absence of reporting on linguistic correctness indicators suggests that authors implicitly assume that LLM-simplified texts are grammatically correct. However, this assumption can be problematic in medical contexts, where ambiguous phrasing or grammatical errors can alter the meaning of health instructions and potentially cause harm to patients. For instance, even minor errors in negation or word order can result in dangerous misinterpretations. Therefore, the failure to systematically assess and report linguistic correctness represents both a methodological oversight and a patient safety concern. Furthermore, the few studies that assessed overall understandability relied exclusively on expert assessments. However, experts such as health care professionals cannot reliably predict what laypeople understand or do not understand. This validation gap reveals a critical conceptual conflation: readability, as measured by formulas, captures surface-level text characteristics such as sentence length, word length, and syllable counts. However, it does not measure whether a reader actually understands the content. By contrast, understandability refers to the extent to which readers can extract meaning from text, apply the information to their own situation, and make informed decisions based on what they read. Consequently, improved readability metric scores do not guarantee that medical content is also factually correct, contextually appropriate, or genuinely understandable to the target group. This distinction is particularly significant in medical contexts, where patients must not only read but also correctly interpret and act upon health information. The complete absence of patient-centered outcome assessment represents the most significant finding of this scoping review.

To the best of our knowledge, this is the first scoping review focusing specifically on automatic text simplification methods for PEMs. This distinguishes our work from previous research on medical AI, which has predominantly focused on text generation [69-78] rather than text simplification or the simplification of clinical records [79-82], rather than PEMs.

Nguyen et al [83] conducted a systematic review and meta-analysis of online PEMs related to cleft lip and palate. In line with our findings, they identified the critical absence of understandability testing, as only 1 study directly assessed patient understanding of simplified texts. In contrast to their disease-specific approach, our review spans a broader range of medical topics, evaluates linguistic quality and content fidelity, and employs a broader search strategy.

In a broader scoping review, Aydin et al [84] examined LLM applications across multiple domains of patient care, including education, engagement, workload reduction, patient-centered health customization, and communication. However, their search was limited to PubMed and was conducted in June 2024. They similarly reported readability improvements in studies addressing automatic text simplification. Their conclusion that LLMs can create accessible materials, help interpret complex information, and enhance patient-provider communication—while also noting that accuracy, readability issues, and ethical concerns require further development—aligns with the findings of our review.

The complete absence of direct testing of understandability with laypeople in all 31 studies is significant, reflecting a systemic methodological limitation in this field: the uncritical acceptance of readability formulas as valid measures of text simplification. These findings are further supported by studies indicating that standard readability formulas have important limitations. They primarily count variables such as sentence length, word length, or polysyllabic words, but do not assess whether the text aligns with the actual comprehension, context, expertise, semantic understanding, and textual coherence of the target audience. Common readability formulas cannot adequately assess the actual understandability of medical texts [85]. Furthermore, readability formulas may judge short technical terms and abbreviations as simple, despite being far less understandable than longer, everyday descriptions [86,87]. For example, the term “HbA_1c” is very short but often unknown, whereas the term “long-term blood sugar” is longer but more self-explanatory. Additionally, in-text explanations of terms are often needed for comprehension; however, they can lengthen and complicate sentences [88].

Based on the findings of this scoping review, this study has several recommendations for clinical practice. First, LLMs should be used as assistive rather than autonomous tools for simplifying PEMs used in clinical practice [51]. This implication is supported by a recent study showing that LLMs can approximate targeted RGLs when simplifying health information materials about psychiatry, but their outputs are inconsistent, with significant variability in reading levels and deviations from the intended content, making them unsuitable for standalone deployment in health care settings [96]. Second, all AI-simplified PEMs must undergo mandatory expert review by qualified health care professionals to verify content fidelity and clinical appropriateness before dissemination to laypeople [38-41,45-49,52,59-61,66]. Third, health care organizations using AI-assisted text simplification tools should establish clear protocols defining quality checkpoints and approval workflows. Continuous monitoring of performance can track both readability improvements and error rates across different medical domains [59].

Although LLMs hold considerable promise for enhancing patient health literacy through simplified text versions, their use in clinical practice requires careful attention to inherent limitations, including the risk of reinforcing biases and stereotypes embedded in training data. The use of AI in health care communication requires an ongoing commitment to the accuracy of generated content [50].

Although this scoping review followed rigorous and systematic methods, some limitations must be acknowledged. First, rule-based technologies were excluded from this review. This decision was made to focus on the contemporary and rapidly evolving landscape dominated by generative AI, which offers greater scalability and adaptability for text simplification. Second, although different prompting strategies were used across the included studies, the quality of these prompts was not evaluated. In accordance with scoping review methodology, which aims to map the extent and nature of available evidence rather than critically appraise methodological quality, a systematic assessment of prompt design was not conducted. Consequently, it was not possible to determine whether the observed failure to meet specific target RGLs reflects model limitations or suboptimal prompting strategies. However, such an analysis could have provided deeper insights into optimal prompt engineering for PEM simplification. Lastly, the database search was conducted in May 2025, and the gray literature search in July 2025. Given the rapid pace of AI development, particularly regarding LLMs, new relevant studies may have emerged since, and some findings may already become outdated shortly after publication. Finally, the included studies were heavily skewed toward English-language PEMs and US-based contexts, limiting the generalizability of the findings, as text simplification approaches may perform differently across languages with varying grammatical complexity (eg, morphology, syntax). Moreover, LLMs are predominantly trained on English data [97], which may lead to lower performance in other languages. Additionally, the readability metrics used (eg, US RGLs) are specific to the US educational system and may not directly translate to plain-language guidelines in other cultural health care systems.

To our knowledge, this is the first scoping review to comprehensively synthesize evidence on automated language processing technologies for PEM simplification, systematically mapping linguistic, quality, and content fidelity outcomes.

The findings of our scoping review indicate that although LLMs can improve the readability of PEMs, they predominantly fail to achieve the recommended sixth-grade RGL. However, the most significant finding is the identification of a critical validation gap: no study has assessed whether laypeople actually understood the simplified PEMs, nor has any study evaluated linguistic correctness. Coupled with variable content accuracy and reliance solely on readability formulas that poorly predict actual understanding by laypeople, these findings have important implications for clinical practice. Currently, LLMs should serve as assistive tools rather than autonomous solutions. All AI-simplified materials must undergo mandatory expert review to verify content fidelity before dissemination to laypeople. Future research must urgently shift from purely algorithmic evaluation to patient-centered validation to directly assess laypeople’s comprehension.