In recent years, there has been a substantive advancement in the use of artificial intelligence (AI) in health care settings. Technological improvements in both natural language processing and machine learning (ML) have positioned AI to revolutionize patient care [1-4]. In nursing, AI can include decision-making tools, educational study materials, patient chatbots, electronic health record notifications, and more [2,5,6]. While AI could enhance nursing practice, it is not without risks. Ethical concerns regarding AI in health care highlight risks in the development, reliability, integration, and confidentiality of these models [3-8]. For 23 consecutive years, nursing has been ranked as the most ethical and honest profession in the United States [9]. Due to this, nursing must be involved in researching, developing, and implementing cutting-edge technologies in patient care, such as AI, to enhance care competence and ensure safety for all persons [10,11].

In the United States, 129 million people are estimated to have 1 chronic illness, and 42% of those individuals have 2 or more chronic conditions [12]. Despite many of these conditions being preventable, the prevalence of chronic illness is expected to increase [13]. The economic impact on health care and the financial toxicity placed on patients are significant [14]. The most prevalent chronic diseases include cardiovascular disease (CVD) and diabetes, which are estimated to cost the United States over 1 trillion dollars annually [13]. While there is movement in the health care field toward upstream preventative measures to mitigate the prevalence and cost of chronic disease, these efforts are limited by structural, social, and environmental factors. Moreover, findings from a longitudinal, multisite, cohort study of 3972 persons living with HIV suggested that a visit to a cardiology clinic was not linked to improvements in CVD risk factors or a decreased risk of major adverse cardiovascular events [15]

Social determinants of health are nonmedical factors (housing, insurance, education, etc.) that contribute to well-being and drive health outcomes [16]. However, sociodemographic factors, such as education, race, health care access, living situation, and income, have been connected to chronic disease prevalence in many populations [12,16-18]. Staggering differences exist when considering that persons from diverse backgrounds have twice the likelihood of death from CVD [14,19] and stroke [20] than their counterparts [13]. This is true for groups with multifaceted and interconnected identities [21,22] as well as older adolescents entering adulthood [23]. These differences may be attributed to minority stress (chronic stress from bias and stigma), varied health care access, and heightened cardiovascular risk [14,24-26].

Persons with HIV are at an increased risk of CVD, which remains a leading cause of death worldwide, with an estimated 32% cause rate in 2020 [13,20]. Risk factors for CVD, such as hypertension and diabetes [20], are also increasing in prevalence, with diabetes now being the eighth leading cause of death [13], particularly in diverse groups [27]. Persons with HIV are at an increased risk of CVD due to chronic inflammation from HIV infection, hyperlipidemia-causing medications, and lifestyle factors [28]. The potential to mitigate these disparities through technologies that have been gaining significant uptake in patient care, such as AI, is vital for advancing health optimization.

Cardiometabolic health is the body’s ability to manage risk factors, such as blood pressure, blood sugar, cholesterol, and weight to reduce the risk of heart disease, stroke, and diabetes [21]. Despite an estimated 1.2 million people living with HIV in the United States [27], there is a limited presence in health care of AI-developed patient education tools for cardiovascular health promotion and behavioral disease prevention. The use of AI in health care has predominantly used ML and deep learning to develop and train prediction models, screening tools [29,30], and patient education adjuncts to help decrease new infections and manage current illnesses [2,4,31,32]. Using deep neural networks to develop ML models can help predict CVD risk in people with HIV [33], identify candidates for pre-exposure prophylaxis [34], and monitor treatment adherence [7]. Furthermore, AI can be used to guide treatment plans, provide personalized reminders, and serve as chatbots for basic questions [7]. Chatbots specifically have been used successfully in sexual health because of their ease of use, availability, and accessibility. However, there are concerns regarding understandability and accuracy [35]. These same issues are found in chatbots for cardiovascular care prevention [36]. ML-based chatbots may exhibit limited conversational understanding, provide inflexible topic options, and present biases and safety risks [37,38]. However, there is promise in developing personalized large language models (LLMs) to mitigate these limitations in areas, such as prostate cancer [39] and ophthalmology [39,40]. While these methods require further development, one of the potentially largest opportunities for AI in health care is in improving patient education using an interdisciplinary lens.

In order to gain a better understanding of AI, it’s important to first comprehend the ways AI is described. Common terms in the understanding and application of AI are defined in Table 1.

Patient readmission continues to be a substantial cost for hospitals and the health care industry [41,42]. While discharge education could reduce readmission rates, there are substantial barriers to successful discharge education, such as limited learning assessments, standardized education forms, and limited nursing time [42,43]. Patients are estimated to spend around 5 minutes with providers and 20 minutes with nursing staff on the day of discharge [44]. It is estimated that fewer than 60% of patients understand their diagnosis after discharge, and less than 44% can recall appointment details [45]. Given the limited understanding of discharge instructions and the use of advanced terms by providers to describe illness [45], it is understandable why patients may be uncertain of how to appropriately integrate health teaching upon leaving the hospital setting. Studies suggest that there may be substantial value in using AI in postdischarge education and monitoring [46-50]. AI-generated discharge summaries are promising for patient understanding and personalization when compared to standardized templates currently used by clinicians [46,47]. However, any AI model is only as good as the data on which it is trained [7].

LLMs are interrelated with AI, ML, and data science. The process of creating an LLM requires obtaining high-quality data, fine-tuning pre-existing models, and evaluating those models against specific metrics to ensure accuracy, usability, and performance [5]. Despite the large potential of LLMs to revolutionize many sectors, including health care, there are substantial risks to their implementation [48-51]. If the materials on which the LLM is trained are biased, the outcomes of the LLM may perpetuate these biases and limit personalization [5,49,50]. To mitigate these harms, it is important to use diverse datasets and bias-mitigation prompts and obtain user feedback [5,51]. In addition, testing for data refusals, offensive autocompletes, and toxic responses can help determine which LLM to fine-tune, with Llama and Mistral potentially performing the best [49]. Additional ethical concerns with LLMs include false information (ie, “hallucinations”), security and privacy risks, natural resource depletion, transparency in algorithm development and function (ie, “black box”), and long-term cultural effects [5,51,52]. These risks are multiplicative when considering intersectional identities and perpetuating minority stress in already vulnerable populations.

This project uses an interdisciplinary team of nurses, public health professionals, and computer scientists to address how intersecting social and structural determinants shape CVD in populations with HIV. The purpose of this tutorial is to describe the iterative steps of developing a novel, intersectionality-based LLM to promote cardiovascular health among persons with HIV. To date, the authors were unable to identify other studies that integrate an intersectional approach into LLM development for enhancing cardiovascular health in persons with HIV.

First introduced by Crenshaw [53,54], intersectionality describes how overlapping marginalized identities can compound experiences of discrimination and oppression. Experiences of discrimination and social stigma create a hostile environment for marginalized groups, which can influence health outcomes, as described in minority stress theory [55,56]. While this theory has been readily used in research, it does not necessarily apply to intersectional identities, nor does it adapt to cultural changes over time [57]. Therefore, both the minority stress theory and intersectionality serve as a basis for the model training and resource selection in our LLM, ensuring that all persons can benefit from its use.

The American Heart Association (AHA) is a leading authority for its research, guidelines, and educational resources related to cardiovascular health and disease prevention. The AHA Life’s Essential 8 was selected to guide this work, given that it is grounded in the evidence, clinically relevant, and uses a multidimensional lens to develop cardiovascular guidelines and resources for both health care professionals and the public [21-23]. The American Heart Association's Life's Essential 8 framework offers a comprehensive approach to promoting cardiovascular health by encompassing 8 primary domains, including diet, physical activity, tobacco use, sleep, weight, cholesterol, diabetes, and hypertension [58]. Each domain is integral to reducing the risk of chronic illness, such as heart disease and stroke. This work maintains the focus on cardiovascular health promotion by using these domains to structure the data collection process and overall project objectives.

To attain our goal for LLM development, interdisciplinary collaboration was crucial throughout each step of the process. Interdisciplinary collaboration differs from multidisciplinary or transdisciplinary approaches as it integrates diverse knowledge bases to achieve a shared goal [59]. The team development process is described in Figure 1, and each step required multiple substeps for completion. The first was to develop our team, which consisted of experts in nursing, public health, and computer science (CS; Table 2). The team had members representing various identities as well as research backgrounds in health disparities research and behavioral health interventions.

To begin, the health care professional (HCP) team collaborated to create study objectives and aggregate resources across the domains. After reviewing the resources, the CS team selected the best methods for scraping and benchmarking. The HCP team periodically reviewed the scraped websites to ensure accuracy. While the CS team processed data and began training the selected model, the HCP team created safety guidelines and sample question and answer pairs. While both teams had their respective activities, our weekly meetings proved to be the most fruitful component of this process. To truly collaborate, a shared language had to be developed between the 2 disciplines, as many terms of art were not mutually understandable. By sharing goals and expectations and defining key metrics, we were able to develop a successful, productive team dynamic. The back-and-forth communication between disciplines was key in training and adapting the model. Expert external consultants were added to the team to test the model and provide insight prior to piloting for end-user feedback.

To develop the dataset, we began by creating objectives for the project and dividing these into primary domains, guided by the AHA Essential 8 (Table 3). Then, we collected publicly available, evidence-based health information on CVD prevention, HIV care, and general health knowledge. We refined our search criteria to include internet-based, patient- and provider-facing sites, putting emphasis on validated sources of literature, such as research papers and governmental websites. By focusing on authoritative websites, such as AHA’s Essential 8, health screening guidelines, official health websites (eg, Centers for Disease Control and Prevention [CDC] and World Health Organization [WHO], and current US legislation, we obtained a robust catalog of resources (Table S1 in Multimedia Appendix 1). The entirety of PubMed was included for current scientific research. Certain resources were specifically added to mitigate bias and promote patient safety. These included harm-reduction strategies, therapeutic communication techniques, mental health screenings, and sexual and gender inclusive data. To mitigate copyright concerns, all data sources used were publicly available and either patient or provider-facing. To integrate intersectional patient experiences, we also sampled publicly available HIV forums to formulate baseline knowledge of patient-expressed needs, concerns, and questions. Further, we gathered targeted materials to address our cardiovascular subobjective regarding populations' experiences with health disparities so that it is useful for every individual. This included current legislature and provider directories to build in political context and safety features.

Data scraping was far more complex than simply downloading a single HTML page. In most cases, it required scraping entire websites, as valuable information was dispersed across multiple pages. In addition, some websites hosted important content in PDF format, which was also scraped. Due to restrictions on automatic scraping by some websites, several of these PDFs had to be extracted manually. Traditional processors (pdfminer, pypdf, etc), HTML parsers (eg, Beautiful Soup), Adobe Toolbox, and human refinement were combined to accurately scrape data sites [60-62]. Once this collection of PDFs and text files was assembled, the data was standardized using Microsoft’s Markitdown tool to convert PDFs into plain text [62]. This tool was chosen following an ablation study comparing various alternatives, including PyPDF [61] and PyMuPDF [60], where Markitdown demonstrated the best performance, as validated by both teams. One major challenge during this conversion process, however, was interpreting tables due to wrong formatting. In fact, without proper formatting, table data could be misleading. As a result, we manually edited these tables to align them with their original formatting or removed them altogether if the information was redundant in the document

To develop the model, we compiled current LLMs available for fine-tuning. The selection process would be based on benchmarked scores for information accuracy and model performance [63,64]. The LLMs selected for benchmarking are listed and briefly defined by our CS team in Table 4. With the necessary resources compiled, we initiated the benchmarking process. For this effort, we focused on models of reasonable size that support local inference, such as BioMistral [65], Qwen2.5 [66], and Meta’s Llama 3.1 [67]. This initial comparison helped us identify models demonstrating strong baseline performance. Then, the CS team curated a specialized fine-tuning dataset to further enhance the model’s behavior. This dataset was generated using GPT-4o [68], which produced multiturn Q&A conversations based on the content gathered from the websites. Instructions were given to GPT-4o to create this dataset with an accessible reading level while avoiding complex medical terminology. Using this dataset, we fine-tuned the selected model via the low-rank adaptation of the LLM fine-tuning method to improve its alignment and overall performance [69]. Finally, we applied reinforcement learning using Group Relative Policy Optimization, with reward signals that incentivize the model to generate responses at or around a fifth-grade Flesch-Kincaid reading level and to avoid the use of technical medical language [70].

There were multiple iterations and methods of measurement for testing the LLM. Overall, we wanted to ensure informational accuracy, interpretability, and safety. To reach these goals, we created a rubric to evaluate model answers (Table S1 in Multimedia Appendix 2) and generated expert-developed sample answers (Table S1 in Multimedia Appendix 3). In addition, we integrated mental health algorithms to prompt screenings and emergency services connections for concerning language on substance abuse, alcohol abuse, anxiety, depression, and suicidal thoughts (Figures S1 and S2 in Multimedia Appendix 4). Furthermore, we based the interpretability of the model on Flesch-Kincaid reading scores and medical jargon usage, with a goal of third- to fifth-grade reading level [71,72]. To assess the LLM, we developed a curated set of expert-generated question and answer (QA) pairs. Half were created by Expert 1 (RR) and half by Expert 2 (TJE), with each expert reviewing and refining the other’s contributions to ensure interrater reliability (Table S1 in Multimedia Appendix 5). This high-quality dataset served as part of the initial training input for the model. Following the initial training phase, we prompted the model to independently generate new QA pairs. These model-generated responses were then evaluated against expert-authored answers. Both experts independently reviewed the model’s output for accuracy, clarity, and alignment with clinical best practices. Model refinement was iterative. Each time the model was updated, a new batch of sample QAs was reviewed by the experts. This process is ongoing until there is a consistent consensus between the model-generated answers and the expert expectations, ensuring that the LLM demonstrates reliable, safe, and clinically appropriate reasoning across a range of question types.

Institutional Review Board approval was obtained from Yale University on January 25, 2025 (2000038443).

We had a total of 868 PDF files, contributing to 7 million tokens and 130 QA pairs. Of note, some of the data sources we used have been removed from publication due to changes in federal research priorities, which did not impact this work but may impact future LLM iterations.

Initially, the readability of the resource text was poor. An average Flesch Reading Ease Score of 26.53 for the selected documents equates to the “Very Confusing” level [72]. This is likely because of the medical reliability of selected websites. The distribution of text sentiment, from -1 (negative opinion) to +1 (positive opinion), varied substantially in our resource material , likely stemming from both risk-focused and prevention-focused sources [73]. Based on these findings, we also created a professional score to identify medical jargon and advanced sentence structure, which combined the percentage of medical terms and complex sentences in the responses. Our goal was to continue fine-tuning our model to reach a third- to fifth-grade reading level while maintaining sentiment and limiting medical jargon.

The initial evaluation metrics chosen were based on n-gram precision. An n-gram is the word-for-word alignment of a text. Bilingual Evaluation Understudy (BLEU) measures how many n-grams match the generated text [74]. For example, BLEU 1 measures a single-word alignment, and BLEU 4 measures matching 4-word sequences, respectively. The Metric for Evaluation of Translation with Explicit Ordering (METEOR) is a similar metric that uses synonyms, paraphrasing, and stemmed matches rather than just word-for-word alignment [75]. The Recall-Oriented Understudy for Gisting Evaluation (ROUGE) is a recall-based overlap score between generated and reference text [76]. These metrics provide a comparison between readily available models to find optimal performance for further fine-tuning. Unfortunately, no pretrained model demonstrated substantial alignment with reference answers (Table 5). This is likely due to a disagreement between our requested response readability and the professionally written reference material. Therefore, Llama-3.1-8B was selected as the base model for its superior prompt-following capabilities, strong multilingual performance in English and Spanish, and natural conversational flow [67]. It also maintains a relatively modest parameter count, thus enabling efficient local computation and reducing environmental resource consumption [67].

Question and answer (QA) pairs were generated by each LLM iteration and sent to health care experts to determine accuracy and interpretability. We then optimized the policy using Group Relative Policy Optimization (Figure S1 in Multimedia Appendix 6). The initial responses were substantially limited and required multiple rounds of fine-tuning before a consensus between expert and model responses was found

Initial LLM responses were long, medically complex, and sometimes provided information that should be left to health care providers. While the fine-tuning process is ongoing, the current responses were shorter and demonstrate improved readability. The examples are listed in Textbox 1.

Fine-tuning (Table 6), including prompt editing, further training, and rewards, improved the accuracy, readability, and jargon use of our Llama-based model. Given the readability goals for our LLM output compared to the professional language of input materials, n-gram agreement scores (BLEU, METEOR, and ROUGE) were determined to be unsatisfactory metrics. Therefore, we developed separate metrics upon which the LLM would be evaluated. For the purposes of fine-tuning, accuracy is defined as agreement with expert-provided sample answers (1 being the lowest agreement and 5 being the highest agreement). Readability is defined as agreement with the advised patient-teaching language metrics, including syllable count, words per sentence, and sentence structure (1 being the lowest agreement and 5 being the highest agreement). Professional score is defined as the level of personalization and bias-free language (1 being the lowest agreement and 5 being the highest agreement). The Kincaid score is a validated readability metric with a goal reading level of 6 for middle school readability [72]. The Jargon score is the quantity of medical jargon used in the text, with a goal of as close to zero medical jargon as possible. We are continuing the fine-tuning process and testing additional questions against expert opinion before obtaining end-user feedback.

The purpose of this tutorial was to describe the iterative steps of developing a novel, intersectionality-based LLM to promote cardiovascular health among persons with HIV. This is one of the first studies to demonstrate the collaborative process between nursing and CS in personalized LLM development, as well as one of the first to specifically investigate AI integration for cardiovascular health promotion and education for people living with HIV. Other studies have investigated AI for cardiovascular and HIV screening, prevention, adherence, and risk prediction [7,33,34]. In addition, AI has been used for treatment plans, discharge summaries, and medical chatbots [7,35,36,46,47,77-79]. However, this work is unique in its intersectional lens, interdisciplinary methods, and varied opportunities for interventional application.

There are a few key takeaways from developing this LLM, collaborating between disciplines, and considering potential applications. First, many resources need to be compiled to have enough tokens for LLM training. Second, gathering experts is vital for developing clinical judgment and managing patient safety. Third, fine-tuning of LLMs should include intersectional considerations, readability, and refusal prompts. Finally, this project highlights the benefit of interdisciplinary collaboration for health care innovation. However, developing communication strategies and shared learning opportunities may be potential methods to facilitate this collaboration. The initial challenges of this process came from inadequate premade LLM performance, high reading levels of sample sources, and difficulties with image and table scraping on websites. However, these challenges were mitigated by integrating expert feedback and fine-tuning for accessibility.

Ongoing fine-tuning is in process to improve the readability and accuracy of our LLM. The LLM will be ready for end-user testing once the readability score consistently produces a fifth-grade level, the expert result rubric shows model-expert consensus, and the bias and screening challenges are successfully managed. After expert testing, end-user testing will evaluate the model’s usability, clarity, and relevance in real-world contexts. This phase will involve patients interacting with the model to pose questions and assess the helpfulness of its responses. Participants will provide structured, quantitative feedback on response quality, trustworthiness, and practical value, as well as identify any gaps or misunderstandings via open-ended, qualitative feedback. This input will be used to guide further model refinement, ensuring that the LLM not only aligns with expert standards but also meets the informational needs and expectations of its intended users. The rationale behind multiple iterations of expert feedback before end-user testing is to limit ethical concerns regarding inaccurate or biased information and the risk of potential patient harm. While bias-free writing references were included in the materials used to train this model [80], ongoing bias and information security testing are needed.

This tutorial outlines our team’s development of an LLM aimed at promoting cardiovascular health among individuals living with HIV. Although the model demonstrates promising performance improvements, several limitations inherent to AI development and health care applications remain. First, the risk of algorithmic bias persists despite comprehensive mitigation strategies. Our approach combined multiple LLM architectures, external expert input, and integration of diverse, reputable data sources, including guidelines from the American Heart Association and insights from patient forums, to build a model that aspires to be broadly representative. Nevertheless, fully capturing the complexity of all user perspectives is challenging, and some residual bias may remain. Ongoing evaluation and refinement are required to further minimize any unintended biases. Second, excessive energy consumption is a notable concern in AI model development. We addressed this by using secure, energy-efficient servers and designing a lightweight architectural solution to reduce resource usage. While these measures contribute to more sustainable operation, wider energy considerations linked to large-scale model deployment remain an area for continued optimization. Third, ensuring the generalizability of the model across a wide range of populations is a key challenge. Although our development process incorporated diverse data sources and expert feedback to enhance the model’s applicability for all users, further validation across additional real-world settings and demographic groups is necessary to ensure that the model performs reliably in various contexts. Fourth, our custom evaluation metrics introduce both advantages and limitations. Building a domain-specific LLM, tailored to cardiovascular health in HIV care, allowed us to outperform general-purpose models on specialized tasks. However, our Accuracy, Readability, and Professionalism scores remain bespoke and carry inherent subjectivity. Experts from nursing, public health, and CS applied the rubric independently and reached consistent judgments, but some subjectivity persists. To anchor our new Readability and Professionalism scales to established tools, we conducted an analysis comparing them against Flesch–Kincaid readability levels and a third-party bias-detection measure, which showed encouraging alignment. Nevertheless, these bespoke metrics are not yet standardized and will need broader revalidation in patient comprehension studies before they can be adopted in other medical domains.

Despite these limitations, our work exhibits significant strengths. First, our weekly collaborative meetings among nursing, public health, and CS experts, including external consultants, ensured clinical rigor, technical soundness, and ongoing bias audits. Second, we structured data collection, prompt design, and QA pairs around the American Heart Association’s Life’s Essential 8 framework, guaranteeing alignment with current, peer-reviewed cardiovascular guidelines. Third, an intersectional and minority-stress lens guided our training data and prompts, reducing stigmatizing language and enhancing cultural relevance for overlapping marginalized identities. Fourth, we developed custom Accuracy, Readability, and Professionalism metrics tailored to patient-education goals, driving outputs toward sixth-grade reading levels and bias-free language, rather than relying on generic natural language processing benchmarks. Fifth, our iterative 4-step fine-tuning pipeline (GPT-4 QA generation, expert curation, low-rank adaptation tuning, and reinforcement learning from human feedback), combined quantitative rewards with qualitative expert review to progressively improve model performance. Sixth, by choosing a lightweight Llama-3.1-8B base model and energy-efficient servers, we minimized environmental impact without sacrificing output quality. Finally, our transparent documentation of resource scraping, benchmarking, and metric development provided a clear blueprint for replication or adaptation in other disease domains. These improvements provide a robust foundation for future efforts to refine the model further and extend its application in managing comorbid conditions across varying clinical populations.

The CARDIO LLM has significant implications for both nursing clinical practice, public health policy, and research. In clinical practice, integration of the LLM into electronic health records could enhance patient discharge education. In addition, our LLM could be adapted for audiovisual platforms to support educational games, virtual or augmented reality scenarios, and telehealth services. Furthermore, there are multiple policy implications for health-related LLM development. These implications include warranting regulations for privacy, implementation in the clinical and community-based settings, ethical use, and consistent validation practices that are grounded in clinical evidence-based guidelines to ensure the safety and reliability of patient-facing AI tools. For research, the technology offers a platform for exploring personalized patient teaching through features like customizable avatars and integration of patient health data, as well as for developing advanced capabilities, such as patient reminders and resource connections. These innovations promise to expand the role of technology in enhancing patient engagement and advancing health care delivery.

The purpose of this tutorial was to describe the development of an intersectionality-based LLM designed to promote cardiovascular health among individuals living with HIV. We found that a fine-tuned, Llama-based model shows promise in delivering personalized, provider-driven, culturally sensitive discharge education based on the American Heart Association’s “Life’s Essentials 8.” Integrating AI into practice requires careful consideration of ethical concerns, data quality, and ongoing expert evaluation to ensure accuracy, reliability, and patient safety. The collaboration between researchers, nursing, and computer scientists highlights the importance of interdisciplinary efforts in creating innovative and effective health care solutions. As we continue to refine and test this model, we aim to create a scalable, sustainable tool that supports equitable health promotion and advances public health efforts. Using a customized LLM to deliver health information can significantly improve patient understanding, support behavior changes, and advance health optimization.