This exploratory qualitative study aimed to explore nurses’ experiences with an LLM-assisted health education workflow in clinical practice. Interviews were conducted after completion of the intervention phase and all outcome data collection of the parent randomized controlled trial (RCT) which investigated an LLM-assisted health education intervention compared with traditional nurse-led education among patients with hematological diseases. During the 6-month intervention period, nurses gained experience implementing the LLM-assisted health education workflow in clinical practice.

All interviewed nurses were recruited from the intervention team involved in the LLM-assisted health education workflow evaluated in this RCT. Patients interacted directly with LLM during the workflow, while nurses supported the process by introducing LLM, providing instructions before use, and reviewing patient-LLM interactions after completion as part of clinical workflow. During this review process, nurses could identify perceived information gaps or safety concerns and provided supplementary education or clarification when needed. These nurses did not participate in real-time patient-LLM interactions. The present study analyzed only nurses’ interview data and did not examine the effectiveness outcomes of the parent trial.

Nurses involved in the LLM-assisted health education workflow evaluated in the parent RCT were invited to participate in the qualitative study. A purposive sampling approach was used, with all eligible nurses who had direct experience with the LLM-supported health education workflow invited to participate. This enabled them to provide firsthand accounts of their experiences, perceived benefits, challenges, and safety considerations. All eligible nurses were invited and all agreed to participate in the interviews.

Sample adequacy was assessed based on the concept of information power proposed by Malterud et al [13]. Information power was considered throughout the recruitment and analysis process. The study had a focused aim of exploring nurses’ experiences with the LLM-supported health education workflow. Participants had high sample specificity because all were frontline nurses with direct experience of the LLM-supported health education workflow. Semistructured interviews provided rich and detailed accounts of participants’ experiences. The adequacy of the sample was further supported by the richness of the interview data and the consistency of identified patterns during analysis. Considering these factors, the final sample of 16 participants was considered adequate to address the research question.

The SSPEC (site-specific prompt engineering chatbot) LLM-assisted patient education system jointly developed and deployed by the Institute of Basic Medical Sciences, Chinese Academy of Medical Sciences, Peking Union Medical College, and School of Basic Medical Sciences, Peking University, serves as the technical platform for this work [14]. The system was designed to provide patient-oriented health education knowledge, using the hfl/chinese-alpaca-2-7b model as the underlying LLM. Detailed information regarding the model is provided in Multimedia Appendix 1.

Semi-structured, in-depth interviews were conducted at the Institute of Hematology and Blood Diseases Hospital, Chinese Academy of Medical Sciences in Tianjin, China, from December 2025 to March 2026, after completion of the RCT and related data collection. To guarantee uniformity, all interviews were conducted by the same researchers, who received in-depth training in qualitative interviewing and were guided by an expert in qualitative methods, including practice interviews prior to data collection. The interviews began with brief questions about participants’ demographic characteristics and previous experience with LLMs and AI-based tools. Participants were then encouraged to freely discuss their experiences with the LLM-supported health education workflow, including perceived benefits, challenges, changes in clinical practice, and safety considerations, in Chinese. Every interview was recorded while taking place in the hospital’s interview room. The audio interviews lasted between 37 and 53 minutes. Field notes were used to record reflections, body language, and facial emotions. Recordings were transcribed verbatim within 24 hours and double-checked by 2 researchers. The written records were supplemented with pertinent field notes. For reporting purposes, selected quotations were translated into English by the research team.

We prepared a semistructured interview guide based on the research objectives to thoroughly examine participants’ experience (detail is provided in Multimedia Appendix 1). To ensure clarity and appropriateness, this interview guide was pretested with a group of clinical nurses and nursing researchers with relevant professional experience who were not part of the main study, and it was modified in response to their opinions.

Interview data were analyzed using an inductive codebook thematic analysis approach, following the phases of thematic analysis described by Braun and Clarke [15]. The interview transcripts were originally analyzed in Chinese. Selected quotations were translated into English and verified against the original Chinese transcripts by the research team to ensure semantic accuracy and consistency. No back-translation or formal cross-cultural validation was conducted, as the translations were used solely for reporting participant quotations. An inductive approach was used to explore nurses’ experiences with LLM-assisted health education. Two researchers independently read the transcripts several times and conducted line-by-line coding to identify initial codes from participants’ descriptions of their experiences with the LLM-assisted health education workflow. Coding was conducted inductively without using predefined categories. The researchers compared and discussed the codes during the analysis process and any differences were resolved through discussion. The initial coding framework was iteratively refined as the analysis progressed. Related codes were compared to identify connections in meaning and codes reflecting related aspects of nurses’ experiences were brought together to develop preliminary themes. The researchers reviewed these themes against the coded data and full transcripts to assess whether they adequately supported by the data and were sufficiently distinct from one another. The themes were further refined through team discussion to establish the final thematic structure. This study is reported in accord with the Standards for Reporting Qualitative Research (SRQR) reporting guideline [16]. The completed SRQR checklist is provided in Checklist 1.

This study used the following quality control techniques based on criteria from Lincoln and Guba [17]: (1) confirmability (researchers engaged in reflexive discussions throughout data collection and analysis to consider their assumptions, interpretations, and potential influences on the findings); (2) transferability (detailed presentation of participants’ backgrounds and the clinical context was used to facilitate assessment of the applicability of the findings to other settings); (3) dependability (discussion and resolution of coding discrepancies were conducted, along with iterative refinement of the coding framework throughout the analysis process); and (4) credibility (researcher discussions and review of preliminary themes took place to ensure that the findings accurately reflected the interview data) [18].

The interviewer was involved in coordinating the parent RCT but was not involved in intervention delivery or outcome assessment. The interviewer received training in qualitative interviewing and maintained a reflexive stance throughout data collection and analysis. No supervisory relationship existed between the interviewer and participants, and nurses were encouraged to share both positive and critical experiences.

This study was approved by the Institute of Hematology and Blood Diseases Hospital, Chinese Academy of Medical Sciences (number: QTJC2025066-EC-1). The parent trial was registered at the Chinese Clinical Trial Registry (registration number: ChiCTR2500107909). The study adhered to the principles of the Declaration of Helsinki [19]. Written informed consent was obtained from all nurse participants before interviews. All interview data were deidentified and participants were assigned unique identification codes during transcription and analysis. Access to the data was limited to the research team. No patient-identifiable information from LLM interactions was included in the qualitative dataset. They were compensated for their participation via leave arrangements and salary support, funded by the research grant. To avoid influencing participation or interview responses, participants were informed of the leave and salary support arrangements only after all interviews had been completed.

Due to confidentiality and privacy concerns related to interview participants, the original interview transcripts are not publicly available. Additional information regarding the study methods may be available from the corresponding author upon reasonable request.

The demographic characteristics of the 16 hematology nurses are presented in Table 1, and the themes and subthemes identified from the interviews are presented in Table 2.

Three key tensions characterized nurses’ experiences: balancing efficiency gains with information safety concerns, role transformation by emerging professional responsibilities, and potential patient benefits with barriers to effective use.

Previous studies have highlighted the practical value of LLMs as clinical assistants, including improving patient education materials and supporting clinical decision-making [20,21]. However, concerns regarding LLM “hallucinations” and inaccurate information remain significant barriers [22-24]. Consistent with prior studies, nurses in our study recognized the potential benefits of LLM-assisted health education while also expressing concerns regarding information reliability and safety. Our findings also further highlighted an important tension between efficiency and safety: while LLMs may reduce repetitive educational work, they also require nurse oversight to support safe and appropriate information delivery [25]. Similar issues have been reported in clinical decision support systems, where the usefulness of digital tools depends not only on the information they provide but also on how they fit into existing clinical workflows and professional practices [26]. These findings suggest that safe integration of LLM into nursing practice depends not only on technological performance but also on clinical workflows for nurse-led verification and correction.

Contrary to concerns that LLMs might lead to overreliance [25], our findings indicate the opposite view: nurses demonstrated heightened awareness of their professional responsibilities, actively seeking to compensate for gaps in their expertise even when LLMs were available as support. LLMs supported professional development but also amplified cognitive demands and accountability, particularly among less experienced staff.

From the nurses’ perspective, LLM use may help patients to access a broader range of health education information and participate more actively in disease-related decision-making. This finding is consistent with previous research by Dehelean et al [27]. However, participants noted that older patients often have limited digital literacy; their reluctance to use LLM is not due to the lack of valuable content but rather insufficient support in effectively using the technology. This aligns with previous studies showing that older adults, retirees, and individuals with lower educational levels are overrepresented among nonusers [10].

Furthermore, although LLMs have been explored for providing emotional support in psychological care and show considerable potential for demonstrating empathy [28], empathy remains a key element of patient care and person-centered practice. Similar concerns have been reported with electronic health records, where documentation demands may affect direct patient communication [29]. This highlights that LLMs should complement rather than replace human interaction in health care settings [30].

Frontline nurses experienced a complex set of tensions when using LLMs for patient health education. The technology enhanced efficiency, consistency, and reach, yet simultaneously increased professional responsibilities, required careful supervision, and exposed specific patient limitations. Beyond documenting general benefits and concerns of LLMs in health care, this study highlights 3 important practice implications: nurses actively assumed a gatekeeping role rather than relinquishing responsibility; less experienced nurses faced disproportionate pressure, suggesting targeted training needs; and patient barriers were driven not only by content limitations but also by insufficient support for technology use.

Our findings can be further interpreted through the lens of the normalization process theory [31], which focuses on how new practices are incorporated into routine clinical work. Normalization process theory was not used to guide data collection or thematic development in this study; rather, it was applied retrospectively as a secondary interpretive framework to contextualize nurses’ experiences with LLM-assisted health education. From the perspective of coherence, nurses considered the value of LLM-assisted health education by weighing its potential to improve efficiency against concerns about information safety. Regarding cognitive participation, nurses described changes in their roles and responsibilities as well as the need for additional knowledge and support. In relation to collective action, nurses highlighted the importance of integrating LLM use into existing education processes while maintaining professional oversight. Reflexive monitoring was reflected in nurses’ reflections on whether this approach was appropriate for different patients and clinical situations.

Our findings suggest that the integration of emerging technologies into clinical practice depends not only on model performance but also on how these technologies align with nurses’ roles and clinical needs. Rather than replacing nursing work, LLMs may change nurses’ roles, with nurses reviewing interaction records, adapting information to clinical contexts, and addressing patient-specific needs. Future implementation efforts should consider workflow-oriented design strategies that support nurse oversight, contextual adaptation, and patient-centered use in clinical settings while future studies should evaluate changes in workflow processes and workload-related outcomes.

Despite recent AI accountability legislation [32], responsibility in clinical LLM implementation remains unclear. Prior research has also shown that although patients have positive expectations of AI, they remain concerned about the accuracy of AI-generated information and who is accountable when errors occur [33]. Traditional liability frameworks hold that clinicians, as the interface between LLM recommendations and patient care, remain the ultimate decision-makers and bear primary responsibility for outcomes [25]. In the context of patient education, this study further extends this perspective by showing that frontline nurses did not abdicate responsibility for education quality; rather, they continued to review, verify, and contextualize LLM-generated content, positioning themselves as the primary supervisors of patient education.

First, results may be impacted by selection and social desirability bias and are subject to the constraints of self-reporting. Participants’ awareness of the research context and prior exposure to LLM-assisted education may have shaped their perceptions and descriptions of their experiences. Second, this study focused only on nurses’ perspectives and did not include patients’ direct experiences or perceptions regarding LLM-assisted health education. Although nurses could provide clinical observations of patient engagement during the workflow, these observations may not fully capture patients’ own experiences and perceptions. Future studies incorporating patients’ own experiences are needed to provide a more comprehensive understanding. Third, although no substantially new insights emerged during later interviews, all participants were recruited from a single hospital. The findings may reflect the specific organizational culture, clinical workflows, and experiences of nurses within this setting, which may limit the transferability of findings to other clinical contexts. Future multicenter mixed methods studies are needed to further explore these findings.

Even in the era of digital health, the core of nursing health education remains accurate and efficient delivery of information [34]. This study suggests that, from nurses’ perspectives, LLM use may support more efficient information delivery while also introducing new considerations related to information safety and clinical oversight. Rather than replacing nurses’ roles, LLM use was perceived to contribute to changes in nurses’ roles and responsibilities, along with increased competency demands. This study also showed potential patients benefits and changes in nurse-patient communication. Future research should further examine how LLMs can be safely integrated into clinical nursing workflows, considering nurse oversight needs, workflow adaptation, and patient characteristics.