This study was approved by the ethics committee of the Chinese People’s Liberation Army General Hospital (S2024-767-01). Given the retrospective nature of this study, the requirement for written informed consent was waived by the ethics committee. No compensation was provided to participants. All data were fully anonymized before being included in the study. The study adhered to relevant ethical guidelines and protected the privacy and confidentiality of all patient records throughout the research process.

This study included 197,337 nursing records from the intensive care unit (ICU) of Chinese People’s Liberation Army General Hospital, covering the period from January 2023 to May 2025. These records consisted of free-text descriptions of time-stamped nursing assessments, interventions, and patient responses. All records were deidentified using a validated deidentification pipeline.

To establish a gold standard dataset for model evaluation, 100 ICU nursing records were randomly selected for expert annotation. Three senior registered nurses, each with >5 years of ICU experience and prior familiarity with SNTs, independently annotated the nursing diagnoses and interventions for all samples. Before annotation, the annotators received a structured briefing on the CCC terminology, including term definitions, hierarchical structure, and mapping principles. Annotation was conducted in accordance with the official CCC terminology definitions and classification guidelines. Interrater reliability among the 3 annotators was assessed using the Fleiss κ, which indicated substantial agreement. The gold standard dataset was established through discussions among the 3 nurses to reach consensus on differing annotations. A representative annotated example is shown in Figure 1.

In total, 1500 nursing text samples were randomly selected from the full dataset to construct a manually annotated corpus. Two nursing postgraduate students, each with at least 3 months of ICU training, independently performed annotation after receiving standardized training based on the CCC terminology guidelines. The task involved identifying nursing diagnoses and interventions mentioned in each text and mapping them to the most appropriate CCC terms. When exact matches were unavailable, corresponding NANDA-I and NIC terms were applied as supplementary standards (Multimedia Appendix 1). In cases of disagreement, a senior ICU nurse adjudicated the final labels. To ensure data separation, the 100 expert-annotated evaluation records and the 1500 manually annotated corpus samples were sourced from distinct patients.

We developed CCC nursing terminology with retrieval-augmented mapping (CNTRAM), a 2-stage RAG framework, to map free-text nursing records to standardized CCC diagnoses and interventions (Figure 2). CNTRAM integrates dense embedding retrieval, retrieval-enhanced prompting, and few-shot LLM guidance to improve mapping accuracy and coverage.

In the first stage, free-text records were segmented into text chunks using Chinese and English punctuation, while the original records were retained as subqueries. Given the small scale of the CCC terminology set and its abundant clinically equivalent paraphrases, we prioritized a fine-grained semantic matching strategy. Specifically, we used the BAAI (Beijing Academy of Artificial Intelligence) General Embedding–small-zh model to embed each subquery, all CCC reference entries, and the full annotated corpus into 512-dimensional dense vectors. Semantic similarity between subqueries, reference entries, and annotated corpora was computed using matrix-based cosine scoring. In detail, the embedding vectors of subqueries and CCC reference entries were L2 normalized to unit norm. Subsequently, matrix multiplication was performed to compute the dot product of these normalized vectors, generating a similarity matrix of dimensions (number of subqueries×number of reference entries), where each element quantifies the semantic similarity between an individual subquery and a reference entry. Retrieval results from all subqueries were aggregated, and the unique entries remaining after duplicate removal were sorted by their similarity scores. To determine the retrieval window, we evaluated retrieval coverage via Recall@k (k=1, 3, 5, and 10). Recall@5 provided the optimal balance between retrieval sensitivity and computational cost and was adopted as the default setting (Multimedia Appendix 2). For each subquery, the top 5 most relevant entries were retrieved separately from both the CCC reference entries and the annotated corpus, merged, and deduplicated to construct context windows.

In the second stage, prompts were generated by combining each subquery with its retrieved context following a predefined RAG template. This template enforces CCC coding guidelines, which incorporate the supplementary NANDA-I and NIC terms, and defines a structured JSON output schema (Figure 3). These prompts were then processed by a locally deployed LLM, which generated structured JSON outputs [24]. The resulting JSON was parsed with regular expressions, cross-referenced against the term database for metadata validation, and formatted into a standardized structure suitable for downstream analysis.

To establish comparative baselines and evaluate CNTRAM’s relative performance, multiple models were implemented, encompassing traditional statistical methods, pretrained language models, and LLMs.

Free-text inputs were transformed into term frequency–inverse document frequency (TF-IDF) vectors [25], and candidate CCC terms were ranked by cosine similarity. This served as a lexical-matching baseline reflecting rule-driven coding.

Contextual embeddings were generated using BERT [20]. Semantic similarity between text segments and CCC entries was calculated via cosine distance. This represented a rule-driven baseline with contextual semantic awareness. Additionally, BERT was fine-tuned on the annotated corpus of 1500 records to provide a supervised baseline for comparison.

Llama3.3-70B (Meta Platforms Inc) [26], Qwen3-14B (Alibaba Group) [27], Mistral-7B (Mistral AI) [28], and DeepSeek-R1 (DeepSeek) [29] were evaluated under zero-shot and few-shot prompting to test their intrinsic ability to align free-text nursing notes with CCC terms based solely on coding rules. Additionally, no-RAG experiments were conducted for each LLM to serve as a comparison.

To examine the influence of contextual retrieval and example-based guidance, 2 prompting strategies were compared under the CNTRAM framework.

First, zero-shot prompting was used; for each generated subquery derived from both the segmented text and the original records, the top 5 most semantically relevant CCC coding rules were retrieved and incorporated into the prompt [30]. No example demonstrations were included.

For few-shot prompting, few-shot examples were selected by retrieving the top 5 most semantically relevant examples for each subquery generated from both the segmented text and the original records [31]. These examples were combined with retrieved CCC coding rules, which were generated following the same approach as zero-shot prompting, to form the final prompt. We further assessed the contribution of few-shot examples without retrieval using 20 static examples in the prompt.

Model performance was evaluated using precision, recall, F₁-score, and intersection over union (IoU), with each nursing record treated as a set of terms. CCC terms and supplementary NANDA-I and NIC terms were included in the evaluation. As a single record may contain multiple clinically related diagnoses and interventions, IoU was additionally used to more comprehensively assess the model’s ability to identify all nursing terms, capturing their combinatorial characteristics. IoU is defined as the ratio of the intersection to the union of the predicted and gold standard term sets.

For a given record i, let G_i denote the gold standard term set and P_i denote the predicted term set. In this notation, “|⋅|” denotes set cardinality, “\” denotes set difference, and “∩” denotes intersection. The evaluation components were defined as follows:

TPi/Gi∩Pi/ ,FPi=/Pi∖Gi/ ,FNi=/Gi∖Pi/

Metrics for each sample were calculated as follows:

and

IoUi=TPiTPi+FPi+FNi

where TP_i is the number of correctly predicted terms in record i, FP_i is the number of predicted terms not in the gold standard set, and FN_i is the number of gold standard terms missed by the prediction. Overall performance was calculated as the macroaverage of each metric across all records.

This dataset contained 197,337 records from 880 patients, representing an older cohort (median age 67, IQR 57-74 years) with a wide range of ICU lengths of stay (median 1 day, IQR 0-6 days), as shown in Table 1.

Interrater reliability was evaluated using the Fleiss κ, which measures the degree of agreement among multiple annotators beyond chance. The results showed substantial consistency for both categories, with κ=0.65 for nursing diagnoses and κ=0.62 for nursing interventions (Table 2). These results indicate that the 3 nurses demonstrated a consistent application of CCC terminology during independent annotation, providing a solid foundation for constructing the final consensus-based reference standard used for model evaluation. In terms of terminology coverage, a total of 65 unique nursing diagnoses were observed in the annotated 100-record test set, corresponding to 36.9% of the 176 CCC diagnosis codes. For nursing interventions, 106 unique codes were present, representing 52.7% of the 201 CCC intervention codes.

The CNTRAM framework demonstrated comprehensive superiority over TF-IDF, BERT, fine-tuned BERT, and zero-shot LLMs in mapping nursing records to standardized CCC terminology by integrating retrieval-enhanced and few-shot prompting. For nursing diagnoses (Table 3), the best-performing model was DeepSeek-R1. It achieved a precision of 0.7909, a recall of 0.7901, an F₁-score of 0.7836, and an IoU of 0.7614. All other approaches showed lower results across all metrics. Among traditional methods and language models, the F₁-score ranged from 0.0268 (BERT) to 0.5253 (Qwen3-14B with RAG+few-shot), and the IoU values ranged from 0.0142 (BERT) to 0.4987 (Qwen3-14B with RAG+few-shot).

For nursing interventions (Table 4), DeepSeek-R1 again showed the best overall performance, with a precision of 0.8453, a recall of 0.8504, an F₁-score of 0.8413, and an IoU of 0.8097. Other traditional methods and language model setups exhibited lower performance, with F₁-score varying from 0.0077 (Mistral-7B with no RAG) to 0.5557 (Qwen3-14B with RAG+few-shot) and IoU values ranging from 0.0071 (Mistral-7B with no RAG) to 0.5072 (Qwen3-14B with RAG+few-shot).

We evaluated the impact of prompting strategies by comparing zero-shot and few-shot setups across all models. Few-shot prompting consistently improved performance for both nursing diagnoses and interventions. For nursing diagnoses (Figure 4), F₁-score across the 4 models ranged from 0.0716 to 0.216 under RAG+zero-shot and increased to a range of 0.4298 to 0.7836 under RAG+few-shot. Corresponding IoU values ranged from 0.0704 to 0.2003 with RAG+zero-shot and from 0.3914 to 0.7614 with RAG+few-shot. For nursing interventions (Figure 5), F₁-score ranged from 0.2744 to 0.4461 with RAG+zero-shot and improved to 0.4299‐0.8413 with RAG+few-shot. IoU values ranged from 0.1907 to 0.3563 with RAG+zero-shot and from 0.3561 to 0.8097 with RAG+few-shot. In both tasks, DeepSeek-R1 with RAG+few-shot achieved the best performance. Ablation results for the no-RAG+few-shot configuration are provided in Multimedia Appendix 3.

In this study, we developed CNTRAM, a retrieval-augmented terminology mapping framework integrating dense embedding retrieval, retrieval-enhanced prompting, and few-shot LLM guidance to automatically identify nursing diagnoses and interventions from free-text nursing records based on standardized CCC terminology. Our findings demonstrate that combining RAG with carefully designed few-shot examples markedly improves mapping accuracy compared with TF-IDF, BERT, fine-tuned BERT, and zero-shot LLMs. Specifically, our CNTRAM framework, based on DeepSeek-R1, achieved substantial performance improvements in both nursing diagnosis and intervention mapping. These results align with growing evidence that retrieval grounding improves the factual consistency of clinical text generation models [32-34].

The core challenge addressed by CNTRAM stems from the semantic characteristics inherent in nursing terminology. Nursing diagnoses are typically abstract conceptual constructs requiring inference, demanding that nurses synthesize multiple clues and arrive at diagnoses through reasoning [35,36]. Many diagnoses represent higher-order concepts that do not explicitly appear in nursing narratives. In contrast, nursing interventions are more concrete, operationalized, and linguistically salient, typically expressed through explicit action verbs such as “monitor,” “turn,” or “assess.” Previous analyses of CCC semantic structure have also noted that intervention concepts are more operationalized and lexically consistent, while diagnostic concepts may overlap semantically and contain broader conceptual boundaries, complicating automated classification [37,38]. These semantic differences likely explain the higher precision and F₁-scores observed in mapping nursing interventions and diagnoses in this study. This also underscores the need for technical approaches that support abstraction and reasoning capabilities during nursing terminology standardization.

On the basis of the professional attributes of nursing terminology, the performance advantages of the CNTRAM constructed in this study are primarily demonstrated through its 3 components. First, the RAG module retrieves contextually relevant CCC entries for each subquery, providing semantic grounding that reduces ambiguity and stabilizes concept alignment [39]. This design is consistent with growing evidence that retrieval augmentation improves reliability in medical question answering and EHR summarization [40]. Second, the 2-stage generation structure enforces strict adherence to CCC coding rules while producing stable JSON outputs that support downstream analysis. Finally, few-shot examples offer essential syntactic and semantic anchors for interpreting abstract or compositionally complex nursing expressions such as “airway clearance impairment,” “aspiration risk,” “infection risk,” “venous catheter care,” and “positioning therapy.” Evidence from biomedical natural language processing similarly shows that combining retrieval with example-guided prompting markedly improves terminology normalization when annotated data are scarce [41].

Beyond task-level performance, CNTRAM maintains methodological continuity with broader concept normalization research while advancing terminology standardization in nursing. Within medical concept normalization, hybrid approaches that integrate rule-based matching, multilevel semantic similarity, machine learning, or multistage LLM workflows have become increasingly prominent for Unified Medical Language System or Systematized Nomenclature of Medicine–Clinical Terms normalization [42-44]. Meanwhile, nursing informatics literature has long emphasized that SNTs enhance interoperability and enable meaningful analysis of nursing patterns [45]. Previous cross-mapping studies demonstrated the feasibility of aligning narrative nursing documentation with vocabularies such as NANDA-I and NIC, although manual mapping workflows proved time consuming and inconsistent across institutions [46,47]. By demonstrating that retrieval-enhanced LLMs can adapt effectively to nursing-specific terminology systems and documentation practices, CNTRAM extends these research directions and offers a practical pathway for automated nursing terminology mapping in real-world environments. Additionally, in this study, the use of LLMs with varying parameter scales showed differing recognition performance. Among them, DeepSeek-R1 outperformed the other models, likely due to its stronger reasoning ability, instruction-following capacity, and better adaptation to Chinese clinical text. This reasoning advantage allows it to handle ambiguous or complex medical and nursing expressions more effectively [29]. Coupled with reliable instruction-following capacity, this ensures accurate mapping within the constrained output schema [48]. Moreover, its specialization in Chinese clinical data further enhances its ability to accurately map nursing terminology, which is essential given the unique linguistic and contextual features of Chinese clinical language [49].

From a nursing practice perspective, CNTRAM offers a scalable method for converting unstructured nursing records into structured, analyzable data. Identifying nursing practice features, such as nursing diagnoses and interventions, within practitioners’ observation records helps validate the value of SNT for enhancing interoperability and the applicability of nursing data [45,50,51]. By identifying patients’ nursing problems and clarifying corresponding nursing actions, CNTRAM can provide a clinically interpretable basis for risk stratification and outcome assessment. Recent observational evidence further supports this direction. Studies from Italian pediatric [52,53] and adult oncology settings [54] have shown that CCC-based nursing data can capture nursing care complexity and are associated with intrahospital and ICU transfers, diagnosis related group–based medical complexity, and length of stay. These findings reinforce the potential of structured nursing data to reflect patient risk and care demand. Accordingly, the CCC-coded information produced by CNTRAM offers a structured foundation that can be incorporated into future analyses of complexity, outcomes, and resource use. In addition, the high IoU score achieved in this study indicates the model’s ability to effectively represent comprehensive sets of nursing diagnoses and interventions, reflecting the combinatorial nature of the nursing process. Notably, nursing diagnoses such as risk of infection and risk of aspiration require comprehensive judgment based on multiple clinical conditions and are prone to certain recognition errors [55]. Generalized nursing interventions, such as clinical measurements and physical examination, may have mapping biases due to unclear classification granularity [56]. Misclassification of nursing diagnoses can affect nurses’ risk stratification assessment of patients, while misclassification of nursing interventions may disturb the calculation of nursing workload and the correlation between intervention frequency and patient risk [57]. In practical application, CNTRAM outputs can serve as an aid for clinical decision support and documentation, but they still require judgment and review by clinical nurses.

However, this study has several limitations. First, the analysis was based on data from a single ICU with a relatively limited dataset, and the test set covered only a subset of commonly used CCC codes. This partial coverage constrains our ability to generalize the findings to the full CCC terminology system. Second, although few-shot prompting improved model performance, it introduced a dependence on manually curated examples. Such reliance may lead to sampling bias. Third, as highlighted in recent health care RAG literature, robust evaluation frameworks are essential to mitigate issues related to bias, trust, and privacy [58]. Future research could further validate the generalization capability of CNTRAM through multicenter and multilingual data verification. Additionally, the flexibility of CNTRAM to adapt to different nursing terminology systems, such as NANDA-I, NIC, and International Classification for Nursing Practice, could be examined. This would involve integrating diverse domain knowledge resources into the RAG framework to enhance semantic granularity and expand the coverage of terminology mapping [59]. Furthermore, embedding CNTRAM into EHR systems for real-time semantic standardization could provide actionable support for nursing practice and clinical decision-making.

In this study, we developed CNTRAM, an LLM-based 2-stage RAG framework designed to map free-text nursing records to the CCC terminology. CNTRAM combines dense embedding retrieval and few-shot prompting. Using DeepSeek-R1 as its backbone LLM, this framework outperformed traditional methods, no-RAG models, and zero-shot prompting approaches. The results demonstrate that the integration of RAG, few-shot learning, and LLMs improves mapping accuracy and offers a feasible solution for mapping unstructured nursing data to standardized terminologies.