Effective physician-patient communication is essential in clinical practice, particularly in oncology, where radiology reports play a crucial role. These reports, often filled with technical jargon, can be challenging for patients to understand, impacting their engagement and decision-making.

Large language models (LLMs), such as GPT-4, offer a novel approach to simplifying these reports, potentially enhancing communication and improving patient outcomes.

Previous studies have demonstrated that LLMs can effectively process and simplify complex medical texts, showing considerable potential in enhancing accessibility and comprehension for patients [1-3]. For instance, research by Amin et al [1] emphasized the role of LLMs in improving the readability of radiology reports through automated summarization, helping patients better understand their medical conditions. Similarly, studies by Doshi et al [2] focused on LLMs’ ability to streamline complex clinical information, making it more digestible for nonexpert users, particularly in the context of patient education. These studies have highlighted the improvements in readability but have largely neglected the application of LLMs in real-world clinical workflows, particularly regarding physician-patient communication and the broader impact on health care delivery, especially in oncology settings. In recent years, more research has begun to address this gap, showing that LLMs can not only enhance comprehension but also improve communication efficiency by reducing the time spent by health care providers on report interpretation and by fostering clearer interactions between physicians and patients [3-5].

However, the application of LLMs in oncology radiology reports remains underexplored, especially regarding their potential to reduce cognitive load and streamline complex diagnostic information for better patient outcomes.

Our research aimed to fill this gap by investigating the feasibility of using GPT-4 to rewrite oncological radiology reports in a way that preserves clinical accuracy while making the content more accessible to patients. We proposed two potential workflows: (1) artificial intelligence report generation with radiologist review or (2) fully automated AI report generation. This study also examines the ethical and legal issues associated with LLMs in the medical field, providing a comprehensive assessment of their application in enhancing physician-patient communication.

By focusing on improving readability, ensuring medical accuracy, and addressing ethical considerations, our research contributes to advancing medical communication practices. The goal is to make complex medical information more accessible to patients, thereby improving their understanding and involvement in their health care.

All procedures involving collection of tissues were in accordance with the ethical standards of the institutional or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This retrospective compliance study was approved by the Ethics Review Committee of Guizhou Provincial People’s Hospital (2024004), the Third Affiliated Hospital of Sun Yat-sen University (B2023074), the Third Xiangya Hospital, Central South University (2024011), and Jiangxi Cancer Hospital (JC2024006). Written informed consent was obtained from individual or guardian participants. To ensure patient confidentiality, all reports were anonymized to remove any personal identifiers before analysis. This step was crucial in protecting patient rights and ensuring compliance with data privacy regulations.

To generate the IRRs, we used a standardized and structured prompt template. The primary goal of this prompt was to simplify the technical language of the reports while maintaining clinical accuracy and ensuring accessibility for nonmedical readers.

The prompt template used for GPT-4 was as follows:

Multimedia Appendix 1 clearly illustrates the transformation of the original radiology reports (ORRs) into IRRs using this prompt template. It presents both the input (ORRs) and the corresponding output (IRRs), demonstrating how complex medical language is simplified into patient-friendly language.

The reports generated by GPT-4 were reviewed and refined by 3 radiologists to ensure clinical accuracy, logical flow, and consistency with professional standards. This iterative process ensured the reports were simplified without compromising clinical relevance.

A total of 70 (10%) radiology reports were selected from a dataset of 698 pathology reports. Reports were selected using the inclusion and exclusion criteria described in the Study Design section. The selected reports represented a variety of tumor types (eg, lung, breast, and colorectal cancer) and varied linguistic structures. Using stratified random sampling, we ensured a representative distribution of tumor types and diagnostic scenarios, which allowed for the development of generalizable templates reflective of real-world practices.

We created a new instrument, the Radiology Report Understanding Level Assessment Scale (Figure 2), to assess the comprehension levels of nonmedical individuals when reading radiology reports. The design of this scale was informed by foundational theories from health literacy and patient comprehension literature, emphasizing that layperson understanding of medical reports is enhanced by clear report structure, simplified terminology, accurate interpretation of imaging findings, and understanding of report conclusions [6-9]. This scale evaluated following 5 dimensions: understanding of report structure, professional terminology, imaging results, report conclusion, and overall understanding. Each dimension used a 2-point scoring system, with a maximum score of 10 points. The scale was pilot-tested to confirm interrater reliability and content validity.

To ensure the reliability and applicability of the scale, 3 experienced radiologists participated in an iterative review process, offering feedback to refine scoring criteria and ensuring that each dimension accurately captured key comprehension aspects for nonspecialist readers. The scale was subsequently pilot-tested with a subset of reports to confirm interrater reliability and content validity, yielding a high level of agreement among reviewers.

We developed a radiology report explanation template (Figure 3) as a structured framework to simplify radiology report content for nonprofessional readers. The template included sections such as imaging type, examined area, major findings, diagnostic explanation, and recommendations. These sections were designed using health literacy principles to translate technical terminology into accessible language, aiding patient comprehension [6,8,9].

In practice, GPT-4 used this template to generate IRRs from ORRs. The IRRs were then independently reviewed by 3 radiologists, who assessed each section’s readability, clinical accuracy, and relevance to ensure consistency with professional standards. This process included iterative feedback and adjustments to refine the IRRs’ clarity and applicability, providing a reliable tool for enhancing patient comprehension in oncology settings.

We developed another instrument, the Artificial Intelligence Oncology Imaging Evaluation Scale (Figure 4), to assess the clinical accuracy, detail, insight, and actionability of GPT-4–generated oncology imaging reports. The design of this scale was informed by principles in diagnostic accuracy and clinical decision-making research, which emphasize the importance of accurate, detailed, and clinically relevant information for effective patient care [2,10]. The scale included 4 dimensions—image interpretation accuracy, report detail level, interpretation depth and insightfulness, and practicality and actionability—with each dimension scored on a 5-point scale to systematically assess each report’s value.

Radiologists independently reviewed each AI-generated report using the scale, with discrepancies discussed to ensure scoring consistency. An interrater reliability analysis showed a high agreement level, confirming the scale’s reliability. This rigorous process ensures that the evaluation of AI-generated reports reflects clinically meaningful criteria, enhancing the potential of these tools in real-world oncology settings.

The study recruited 30 volunteers, all of whom were family members of patients, ranging in age from 20 to 67 years, with diverse educational backgrounds (junior high school to master’s degrees). None of the participants had a medical background, ensuring their perspectives closely resembled those of typical patients (Multimedia Appendix 2).

Simulated physician-patient interactions were conducted to evaluate communication efficiency. These interactions involved trained physicians explaining the radiology report findings to volunteers after they had read either the ORRs or the IRRs, with communication times meticulously recorded under controlled conditions. This set-up was designed to replicate real-world scenarios as closely as possible while adhering to ethical considerations. The evaluations were spaced 1 month apart to mitigate recall bias and ensure independent assessments of both ORRs and IRRs.

The scales and questionnaires used for readability and comprehension assessment were developed based on health literacy principles and validated through a three-phase process [6,11,12]. First, 3 experienced radiologists independently reviewed the scales for clinical relevance, comprehensiveness, and clarity. Discrepancies in feedback were addressed during consensus meetings, and the instruments were iteratively refined. The finalized scales were then pilot-tested with 10 participants to confirm their applicability and appropriateness.

To assess the fidelity and quality of the IRRs relative to the ORRs, a GPT-4-generated consistency evaluation scale was used (Figure 4). This scale evaluated the IRRs across 4 dimensions: image interpretation accuracy, report detail level, interpretation depth and insight, and practicality and actionability. Each report was reviewed by 3 radiologists with >10 years of experience, ensuring high interrater reliability. Readability was assessed using the Chinese Readability Index Explorer 3.0, a validated tool designed to evaluate the complexity and difficulty of Chinese texts, including vocabulary, sentence structure, discourse coherence, and cognitive load [13]. This tool provided a multidimensional evaluation of the readability differences between ORRs and IRRs.

Comprehension was evaluated using the Radiology Report Understanding Level Assessment Scale (Figure 2), developed specifically for this study. The scale assessed 5 key dimensions: understanding of report structure, professional terminology, imaging results, report conclusions, and overall application to health conditions. Each dimension was scored on a 0 to 2 scale, with a maximum score of 10 points. On the basis of their total score, participants were categorized into 3 comprehension levels: level C (low comprehension, 0-4 points), level B (moderate comprehension, 5-7 points), and level A (high comprehension, 8-10 points). The number of medical terms in the ORRs and IRRs was manually counted by identifying and categorizing medical terms based on a predefined list of common medical terminology used in radiology reports [14,15]. This list was developed by consulting relevant medical dictionaries and radiology resources [15,16]. Two independent reviewers conducted the term identification process to ensure consistency and accuracy.

The reading time for both ORRs and IRRs was measured using a timed reading procedure. Participants were asked to read each report either aloud or silently, based on their preference. The time was recorded using a stopwatch or digital timer, starting from when the participant began reading the report until they finished. The total reading time for each report was recorded and averaged across all participants to calculate the mean reading time for both report types.

A simulated study was conducted to evaluate physician-patient communication, where volunteers were asked to read and interpret radiology reports. In the first round, the volunteers were provided with ORRs, and in the second round, they read IRRs. The volunteers were then asked to explain the patient’s condition to a physician based on what they had read.

Data are presented as the mean and SD. Comparisons between groups were made using ANOVA to evaluate differences between the ORRs and IRRs. The Tukey honestly significant difference post hoc test was used to identify which specific groups differed significantly from each other. This method was selected to control the type I error rate during multiple comparisons. The 95% CIs were reported for point differences to assess the precision of the findings.

The SD for key measures, such as reading time, reading rate, and comprehension scores, was calculated and presented. CIs for the point differences between ORRs and IRRs were also reported to provide additional information on the variability of the results. These changes were made to ensure that the statistical methods were more transparent and the conclusions were robust.

For measures of reliability, Cohen κ was used for categorical evaluations, and intraclass correlation coefficients (ICCs) were computed for continuous measures [17,18]. Cohen κ values and ICCs were interpreted according to established thresholds: values of 0 to 0.2 were considered poor, 0.21 to 0.4 fair, 0.41 to 0.6 moderate, 0.61 to 0.8 substantial, and 0.81 to 1 excellent.

P values <.05 were considered statistically significant.

Between October 2023 and December 2023, the study included 698 patients for analysis. The cohort consisted predominantly of female patients (408/698, 58.45%), as detailed in Table 1. The age range of the participants varied significantly, with the youngest aged 24 years and the oldest aged 82 years, resulting in an average age of 55.27 (SD 12.66) years. A substantial majority of the patients (598/698, 85.67%) were aged <65 years.

As presented in Table 2, the mean word count of the ORRs was 818.74 (SD 197.57). Furthermore, the examination revealed an average of 17.22 (SD 4.01) medical terms within ORRs across all studied malignant tumor categories (Table 3). In addition, by using the IRR template generated by GPT-4 (Figure 3), we structured all the ORRs to maintain consistency in the output framework. It was observed that across all malignant tumors, the average word count for IRRs was 1025.82 (SD 42.87).

A comparison of the word count for ORRs and IRRs (Figure 5; Table 2), for all types of malignant tumors showed that the average word count for IRRs was more than that for ORRs, with this difference being statistically significant (P<.001).

The outcomes of this evaluation, as adjudicated by the 3 radiologists—identified herein as radiologist X, radiologist Y, and radiologist Z—revealed no statistical significance in their appraisals across the defined dimensions (Tables 4 and 5). Remarkably, all dimensions consistently garnered scores of 4 or above, with dimension B notably achieving a unanimous score of 5 (Table 4). This underscores the evaluative framework’s effectiveness in ensuring the IRRs’ adherence to the ORRs insights.

ORRs and IRRs were read independently by volunteers, with the two types of reports being read one month apart, and the reading time and reading rate were recorded (Figure 5; Tables 6 and 7). It was observed that the average reading time for ORRs for all types of malignant tumors was 674.86 (SD 134.08) seconds. Furthermore, for all types of malignant tumors, the average reading time for IRRs was 589.92 (SD 42.21) seconds. A comparison of the reading times for ORRs and IRRs, for all types of malignant tumors showed that the average reading time for IRRs was shorter than that for ORRs, with this difference being statistically significant (P<.001).

It was observed that the average reading time for ORRs for all types of malignant tumors was 72.15 words per minute. Furthermore, for all types of malignant tumors, the average reading time for IRRs was 104.70 words per minute. A comparison of the reading rates for ORRs and IRRs, for all types of malignant tumors showed that the average reading rates for IRRs was faster than that for ORRs, with this difference being statistically significant (P<.001).

The readability of ORRs and IRRs was assessed using the Chinese Readability Index Explorer 3.0. Detailed results are provided in Multimedia Appendix 3. The key findings of the assessment were as follows: for vocabulary, the advanced vocabulary ratio decreased from 18.5% (ORRs) to 3.2% (IRRs). The Lexical Diversity Index reduced from 0.75 in ORRs to 0.45 in IRRs. For sentence structure, the average sentence length was reduced from 32 words (ORRs) to 18 words (IRRs). The complex sentence ratio dropped from 45% in ORRs to 12% in IRRs. For discourse coherence, the coherence score improved from 70 (ORRs) to 85 (IRRs). The average paragraph length decreased from 220 words (ORRs) to 140 words (IRRs). For cognitive load, the information density index was reduced from 0.72 (ORRs) to 0.45 (IRRs). The complex vocabulary density decreased from 18.5% (ORRs) to 4% (IRRs).

Comprehension of ORRs and IRRs showed significant improvements across all evaluated dimensions (Figure 2; Table 8).

The average communication time for ORRs across all cancer types was 1116.11 (SD 57.87) seconds, whereas for IRRs it was significantly reduced to 745.30 (SD 85.08) seconds (P<.001). These results indicate that IRRs not only enhance patient understanding but also streamline the communication process, making it more time efficient. The simulated physician-patient communication workflow demonstrated significant differences in communication efficiency between ORRs and IRRs.

The understanding of report structure score increased from 0.8 (SD 0.4) ORRs to 1.8 (SD 0.2) IRRs (P<.001). The understanding of professional terminology score rose from 0.7 (SD 0.3) ORRs to 1.7 (SD 0.3) IRRs (P<.001). The interpretation of imaging results score improved from 0.6 (SD 0.5) ORRs to 1.7 (SD 0.2) IRRs (P<.001). The understanding of report conclusion score increased from 0.9 (SD 0.3) ORRs to 1.9 (SD 0.1) IRRs (P<.001). Similarly, the overall understanding and application score rose from 0.9 (SD 0.4) ORRs to 1.8 (SD 0.2) IRRs (P<.001). The total comprehension score increased significantly from 5.51 (SD 0.89) ORRs to 7.83 (SD 0.99) IRRs (P<.001). Detailed results are provided in Table 8.

To assess the reliability of these evaluations, Cohen κ values were estimated and they ranged from 0.76 to 0.85 for categorical evaluations, indicating substantial interrater agreement. For continuous measures, ICCs ranged from 0.82 to 0.91 across comprehension dimensions, reflecting excellent interrater reliability (Multimedia Appendices 4 and 5).

After volunteers finished reading the ORRs, the physician engaged in simulated physician-patient communication with the volunteers to explain the patient’s condition and recorded the communication time (Figure 5; Table 9). Across all malignant tumors, the average communication time was 1116.11 (SD 57.87) seconds. In addition, after volunteers finished reading the IRRs, the physician conducted simulated physician-patient communication based on the report content, explained the patient’s condition, and recorded the communication time. Across all malignant tumors, the average communication time was 745.30 (SD 85.08) seconds. Further analysis revealed that, regardless of the type of malignant tumor, the communication time following the reading of the ORRs was significantly longer than the time following the reading of IRRs (P<.001).

AI has influenced every aspect of medical practice [19-22], with recent applications of LLMs further enhancing clinical diagnostic and treatment processes [2,3,10,23,24]. This study demonstrates the considerable potential of AI in improving the understandability of medical information by simplifying oncological radiology reports using GPT-4. Our research confirms that LLMs can effectively transform complex radiology reports into formats that are more accessible to patients, significantly improving both the efficiency of physician-patient communication and patient comprehension. Specifically, we found that IRRs generated by GPT-4 reduced communication time between physicians and patients while enhancing patients’ understanding of the report content.

Our study extends existing research by not only simplifying radiology reports but also standardizing and structuring the IRRs through the use of GPT-4 [1]. In addition, the diverse characteristics of the 30 volunteers, ranging in age from 20 to 67 years and with educational backgrounds from junior high school to master’s degrees, ensured that the reports were accessible to a wide demographic. Despite an increase in word count, reading rates and comprehension significantly improved. To minimize bias, the evaluations of ORRs and IRRs were conducted 1 month apart.

We observed that in simulated physician-patient interactions, the use of IRRs led to shorter interactions and improved patient comprehension. Although a 1-month temporal separation between the 2 rounds was introduced to minimize potential learning effects, it is possible that some residual influence from the first round of explanations persisted. However, we believe that the impact of this effect was minimal and does not compromise the validity of the study’s findings [25].

We also acknowledge the potential for bias due to the lack of blinding, as both physicians and volunteers were aware of whether they were using ORRs or IRRs [26,27]. To mitigate this, physicians adhered to a standardized script to ensure consistency in explanations, and volunteers were kept unaware of the study’s hypotheses. In addition, objective metrics, such as reading time and comprehension scores, were used to provide an unbiased and reliable evaluation of the communication process, further strengthening the robustness of our conclusions [26,27].

For future studies, we plan to implement stronger blinding techniques and explore crossover designs to minimize bias further. In addition, the integration of LLMs in health care introduces important ethical and legal considerations [28-32]. Issues surrounding patient data privacy, AI bias, and the legal implications of AI-generated medical reports must be addressed. Patient data used to train AI systems must be handled securely, adhering to privacy laws, such as Health Insurance Portability and Accountability Act [29]. Moreover, biased datasets could lead to inaccurate medical recommendations, highlighting the need for diverse, representative datasets [30]. The legal ramifications of using AI-generated reports are also critical, as errors in diagnosis or treatment could lead to malpractice claims [31,32]. Therefore, health care providers must remain responsible for final medical decisions, ensuring that AI tools complement, rather than replace, clinical judgment [32].

This study has several limitations. First, most radiology report conclusions are negative, and this study only included malignant tumor reports, limiting its applicability to negative findings. Nonetheless, we believe that with appropriate adjustments, GPT-4 has the potential to improve communication efficiency even for reports with negative findings. Second, while GPT-4 generated reports currently require review by radiologists or clinicians, the time needed for such reviews is significantly less than that required for direct physician-patient communication. As the model continues to iterate, its accuracy is expected to improve, further reducing the workload for health care providers. Third, although the standardized prompt effectively simplified the reports for a diverse group of nonmedical readers, we acknowledge that a uniform approach may not fully address the needs of individuals with varying educational or cultural backgrounds. Future research should explore personalized prompt adjustments to generate reports tailored to specific patient or family member characteristics. Fourth, as discussed in the Introduction section, this study focused exclusively on radiology reports related to malignant tumors. While the findings provide important insights into the accuracy and applicability of GPT-4 for analyzing these specific reports, it is important to recognize that the model’s performance may vary in different contexts. For example, GPT-4 might perform differently in reports involving negative or benign findings, as the model’s training and optimization are heavily influenced by the presence of clinically significant conditions, such as malignant tumors. The language, terminology, and diagnostic challenges associated with benign findings are distinct from those related to malignancies, which could affect the model’s ability to detect and interpret subtle or less critical abnormalities. In addition, the prevalence of normal findings in these reports may introduce different linguistic patterns that GPT-4 might handle with less accuracy, depending on its exposure to such data during training. Therefore, it is essential to explore how GPT-4 can be adapted to handle a broader range of radiology reports, including those with negative or benign findings, in future studies. This would provide a more comprehensive understanding of the model’s potential applications across diverse diagnostic contexts.

Looking ahead, future studies should aim to develop more advanced models that account for the complexity and nuances of medical reports. In addition, it is crucial to assess patient trust and acceptance of AI-generated reports. Another promising direction is leveraging LLMs for personalized medical information delivery, tailoring content to patients’ unique backgrounds and preferences to further enhance comprehension, engagement, and satisfaction.

Our study underscores the potential value of using LLMs to simplify radiology reports in enhancing physician-patient communication efficiency and patient understanding. As we further explore the application of these technologies, we anticipate not only ensuring the quality and efficiency of medical care but also better meeting the personalized needs of patients, fostering continuous innovation and development in the medical field.