The transformation of heterogeneous health care data into the OMOP CDM represents a fundamental challenge in enabling large-scale digital health research. A multilayered architecture that combines semantic matching, LLMs, and a user-friendly web interface was developed to address this challenge and automate the conversion process.

Our system interfaces with multiple clinical data sources, including trial databases, electronic health records, and registry data. At its foundation, OHDSI’s Vocabularies Repository Athena serves as the reference resource for standardized vocabularies. The architecture builds upon established semantic frameworks [18,19] by leveraging the Unified Medical Language System to generate comprehensive concept profiles [20,21].

The architecture uses two complementary technical approaches for semantic processing:

Our implementation uses a three-tiered semantic matching approach:

The system uses several optimization strategies to handle multiple concurrent users and ensure efficient processing:

The web interface, developed in React (Figure 2), provides three pathways for data input:

Upon completion of the mapping process, the system generates structured CSV outputs containing form and field identifiers, field annotations with concept IDs, similarity scores, matched standardized terms, and domain information. This interface design supports both individual researchers mapping specific terms and research teams processing complete datasets, making the system accessible to users with varying technical expertise levels.

We designed a testing protocol to evaluate system performance using a pilot set of 76 NIH HEAL chronic pain and opioid use disorder clinical trial common data element (CDE) terms (Multimedia Appendix 1). Each term was processed through the system, and the resulting mappings were independently reviewed by clinical informaticists to establish a gold standard for accuracy assessment. Our evaluation methodology included binary classification model assessment measuring precision, recall, and F₁-scores, along with usability testing with clinical researchers to validate workflow efficiency and interface accessibility (Table 1).

Our validation approach used 2 distinct datasets to ensure robust evaluation. For development, we selected 76 NIH HEAL Initiative clinical trial CDEs covering domains including demographics, education, employment status, and pain assessments. These elements were chosen to represent diverse data types commonly found in clinical trials. In establishing a gold standard for accuracy assessment, 2 clinical informaticists independently reviewed all mappings, with discrepancies resolved through consensus discussions.

The system’s vector embeddings were generated using the GPT-3 model and were stored in MongoDB collections with optimized indexing for similarity calculations. The embedding process preserves semantic relationships while enabling efficient computational processing. Additional validation was performed using a separate dataset of electronic health record concepts from the NIH N3C COVID-19 enclave, which provided real-world validation in a different clinical domain. Performance was evaluated across similarity thresholds ranging from 0.85 to 1.0, providing a robust assessment of the model’s classification capabilities under varying stringency requirements.

A 2-tiered verification approach was used to validate the successful generation of structured CSV outputs. First, the system was designed to export all displayed tabular data along with supplementary metadata from our Athena-derived database, including concept identifiers, domain classifications, class identifications, and standardized vocabulary references.

The integrity and completeness of these exports were verified through systematic spot-checking against the authoritative OHDSI Athena web repository. For a random sample of 20% (16/79) of the generated concept mappings, we manually compared the exported metadata fields against their corresponding entries on the Athena website to confirm accurate data transfer and representation. Complete success was defined as 100% concordance between our exported metadata and the authoritative source for all sampled entries.

For the similarity scores included in the exports (calculated using cosine similarity between vector embeddings), we verified the mathematical correctness of the calculations by comparing a subset of scores against independently calculated values. While these scores primarily serve as guidance for users selecting between potential matches rather than as absolute metrics of accuracy, we validated that the system consistently produced the expected similarity values based on the underlying embedding comparisons.

We relied on established literature documenting the substantial time requirements for manual OMOP mapping processes to assess the direct clinical impact of our system, particularly regarding time efficiency [10,23,24]. Previous studies have demonstrated that manual mapping of clinical concepts to standardized vocabularies typically requires significant informatics expertise and can take significant time for comprehensive dataset mapping, not assessing the time necessary for clinical-technical team dyad interactions [10,23,24].

While we did not conduct direct comparative timing studies, our expert reviewers (MCBA and RWH) provided subjective estimates of time savings based on their experience with manual OMOP mapping processes. These estimates considered factors including dataset size, concept complexity, and domain specificity. For evaluating the usability of our system from the perspective of end users, we used the System Usability Scale (SUS), a widely used and validated tool for assessing the perceived usability of software and other digital health systems [25]. SUS scores range from 0 to 100, with higher scores indicating better usability and a score of 68 considered the average benchmark [25].

This study involved the development and validation of a computational tool using LLMs to automate the transformation of clinical data into the OMOP CDM. This work builds upon data and processes developed under the Wake Forest IMPOWR Dissemination Education and Coordination Center (IDEA-CC) project [26,27]. While this study focuses on the development and validation of a software tool and does not directly involve interaction with human subjects, the parent project, IDEA-CC, does involve human subject research. The Wake Forest School of Medicine Institutional Review Board has reviewed and approved the IDEA-CC project (IRB00080548), with approval activated on April 2, 2022, and is currently active. This manuscript does not contain any images that could identify individual participants or users.

Using the pilot set of 76 clinical trial CDE terms, our model achieved an AUC of 0.9975 (Table 1), indicating high accuracy in identifying correct mappings. Performance metrics were evaluated across similarity score thresholds from 0.85 to 1.0, with precision ranging from 0.92 to 0.99 and recall scores between 0.88 and 0.97. F₁-scores remained consistently above 0.9 across all tested thresholds, demonstrating robust performance even as classification cutoffs varied.

Analysis of mapping performance across different types of clinical terms revealed distinct patterns of accuracy:

The technical implementation demonstrated efficient processing capabilities across different usage patterns:

This study presents a novel tool that leverages LLMs to automate the transformation of heterogeneous clinical data into the OMOP CDM, addressing a critical barrier in digital health research. The high accuracy demonstrated by our feasibility model, with an AUC of 0.9975, suggests that LLM-based approaches can effectively support the complex task of semantic matching in clinical data harmonization. The development of a user interface accessible to clinical researchers without extensive informatics expertise represents a significant step toward widening participation in large-scale digital health research initiatives.

The implications of this work extend beyond technical performance metrics to address fundamental challenges in the digital health ecosystem. As health care data continue to expand across modalities, shifting from traditional clinical trials and electronic health records to mobile health devices and patient-reported outcomes [1], the need for efficient data harmonization becomes increasingly critical. Our tool’s ability to process diverse data types through semantic embedding demonstrates its potential to support this growing complexity.

Our LLM-based tool successfully automates the mapping of clinical data elements to OMOP concepts with high accuracy (AUC 0.9975). The 3-tiered semantic matching approach effectively captures relationships between research-level data representation and standardized vocabularies, enabling broader participation in large-scale data harmonization initiatives.

Previous data harmonization approaches have relied primarily on exact string matching or manual mapping processes requiring specialized informatics expertise. Our implementation advances this field by leveraging LLM-based semantic similarity, significantly reducing the technical expertise required [28]. While tools such as Usagi (OHDSI team) provide valuable mapping capabilities, our implementation uniquely combines embedding-based similarity with a researcher-focused interface designed specifically for clinical research workflows. The system’s ability to handle both semantic content and meta-information represents an advancement in automated OMOP mapping, particularly in processing complex clinical trial data [10].

Traditional OMOP conversion approaches typically require substantial technical expertise and resources, making them feasible primarily for large-scale institutional projects [6,7]. Our tool addresses this gap by enabling smaller research teams to participate in data standardization efforts. This democratization of OMOP conversion could unlock significant value from currently siloed clinical trial and study data, particularly from smaller research initiatives that would otherwise lack resources for data transformation projects.

The system demonstrates strong performance across diverse data types and clinical domains. The user interface design enables clinical researchers without extensive informatics training to participate directly in data standardization processes, addressing a critical gap in current approaches. Despite these strengths, several limitations remain. First, while our implementation significantly reduces technical barriers, some informatics knowledge remains necessary for extracting, transforming, and loading processes. We are addressing this through companion tools that automate table construction and data transformation steps. Second, the current architecture exhibits performance constraints due to memory requirements for large embedding collections. Future iterations will implement optimized vector storage solutions and approximate nearest neighbor algorithms to enhance efficiency. Third, our validation, while robust, has focused primarily on specific clinical domains (pain, opioid use, and COVID-19). Broader validation across additional domains would strengthen confidence in the tool’s generalizability.

Several promising avenues for enhancement emerge from this work. First, the implementation of specialized vector database technologies could significantly improve performance compared to our current MongoDB-based approach. Second, approximate nearest neighbor algorithms [29] would optimize similarity calculations for larger vocabulary sets. Finally, expanded validation studies across diverse clinical specialties would further demonstrate the tool’s utility across the healthcare research spectrum.

Our LLM-based tool provides an accessible and efficient means to map and convert diverse clinical data into the OMOP CDM, enabling broader utilization of OHDSI’s analytic capabilities and fostering collaborative research using standardized FAIR data. The system’s high accuracy in concept mapping, demonstrated through both exact matches and semantic similarity scoring, addresses a critical need in the digital health research community.

The combination of embedding-based semantic matching and user-friendly interface design addresses both technical and practical barriers to data harmonization. By achieving precision scores ranging from 0.92 to 0.99 across different types of clinical terms, the system demonstrates reliable performance in automated concept mapping while maintaining the flexibility needed for diverse research contexts.

The tool’s successful processing of varied data types ranging from clinical observations to demographic information supports its potential utility across multiple research domains. This aligns with the goals of the NIH HEAL Initiative Data Ecosystem [26,27,30-34] and the NIH Office of Data Science Strategy to make biomedical research data FAIR [5].

By lowering technical barriers to data harmonization while maintaining high accuracy standards, this tool could accelerate the transition toward a more connected digital health research ecosystem. The ability for clinical researchers to directly participate in data standardization processes without extensive informatics expertise could lead to more comprehensive and nuanced data transformations, ultimately supporting more robust research findings.

Future work should focus on implementing performance optimizations; expanding validation across additional clinical domains; and further reducing the technical expertise required for complete extract, transform, and load processes. These improvements will help maximize the tool’s impact on accelerating clinical data interoperability and reuse, particularly for smaller research teams and institutions that may currently lack resources for comprehensive data standardization efforts.