Best Practices for Data Modernization Across the United States Public Health System: Scoping Review

Introduction

Background

The need to modernize public health data systems is more critical than ever. Data modernization goes beyond updating old technology; it fundamentally transforms how organizations manage, analyze, and leverage data. As the amount and complexity of data grow, outdated systems limit an organization’s ability to innovate and respond quickly to new challenges. Thus, continuous and accelerated efforts are crucial to fully realizing the vision of a learning health system [1]. By creating modern and secure data systems, organizations can not only enhance their operational efficiency but also better position themselves to adapt to future technological changes [2].

Modernized data systems can significantly improve the accuracy and speed of data collection and analysis, leading to better health outcomes. In times of crisis, real-time data enable rapid interventions that can prevent adverse outcomes. Importantly, given the serious implications for population health and policy, this transformation plays a pivotal role in advancing public health outcomes (Figure 1). It is essential that our data systems are designed to be inclusive and equitable. By doing so, we ensure that public health initiatives not only reflect accurate data but also promote justice and positive change at the population level [3-6].

‎

In response to the evolving landscape of public health challenges, the Centers for Disease Control and Prevention (CDC) stated that there is a pivotal shift toward enhancing data systems across US public health agencies [7]. The CDC has embarked on a comprehensive data modernization initiative, aimed at integrating advanced data analytics to effectively manage health-related data and improve the timeliness, accuracy, and utility of public health information systems [8,9]. As a result, the 2024 and 2025 milestones for the Public Health Data Strategy were crafted with contributions from over 100 national partners. These evolving milestones are closely monitored and updated to reflect ongoing progress, with success hinging on effective collaboration among state, tribal, local, and territorial public health departments, health care partners, and federal agencies [8].

There are many strategic initiatives on data modernization such as Health Level Seven International, Fast Healthcare Interoperability Resources, and the North Star Architecture, along with the efforts of the Office of the National Coordinator for Health Information Technology, which are pivotal in advancing public health data modernization in the United States [8,10-12]. To further enhance public health responses, continued improvements shaped by stakeholder feedback are essential. For these, the 2024‐2030 Federal Health IT Strategic Plan aims to promote health, enhance care delivery, accelerate innovation, and connect the health system through integrated health data [2].

The COVID-19 pandemic exposed stark vulnerabilities in our public health infrastructure, particularly regarding the obsolescence of data systems [13]. Throughout the crisis, health departments faced significant challenges in their data management capabilities, struggling with the collection, analysis, and dissemination of crucial information necessary for an effective response [14]. Often, different jurisdictions had inadequate and unreliable reporting, ambiguous timelines, difficulties in accessing data, and shifting interpretations from scientific institutions, making it challenging to leverage data for timely public health decisions. This situation underscored the critical need for a comprehensive modernization of public health data systems across the nation to ensure that data can be rapidly accessed, accurately analyzed, and effectively shared to better manage health emergencies and improve overall public health outcomes. While some papers have explored individual aspects of public health data systems, they are often limited to specific sectors or themes. This review offers a more comprehensive perspective by bringing together a wide range of data modernization efforts across the US public health landscape, aligned with national strategies and priorities. It addresses gaps in the existing literature and contributes meaningful insights for future research and practice. We have the mapping needed to mobilize health departments for seamless data sharing and state-of-the-art analytics. With investments in thoughtful data interoperability, governance, and training, public health can be brought into the modern data era. Ultimately, lives depend on having quality data in the right hands at the right time.

Objectives

This scoping review aimed to address data modernization–associated challenges, initiatives, and progress in public health data systems that integrate advanced technologies and standardized practices. By identifying these challenges and best practices, the aim of this scoping review is to provide a comprehensive overview of the current state of public health data modernization, offering insights and recommendations to guide future efforts in creating a more efficient, equitable, and responsive public health infrastructure.

Methods

Search Period and Strategy

The search was conducted from January 10, 2024, to April 15, 2024, with no studies published after April 15, 2024, included in the review. The review was conducted in accordance with the 5-stage methodological framework developed by Arksey and O’Malley [15], which included (1) defining the research question specifically around challenges and best practices in data modernization; (2) identifying relevant research articles that discuss both successes and shortcomings; (3) selecting studies that provide comprehensive insights or innovative approaches; (4) organizing the data to highlight key themes and gaps; and (5) compiling, reporting, and summarizing the findings to outline clear paths forward and areas that need attention (Table 1).

The search strategy covered several bibliographic databases including PubMed, Scopus, CINAHL, and PsycINFO. Supplementary searches were also conducted using Google Scholar to identify additional peer-reviewed literature. The search terms used represent associated keyword combinations of public health, data modernization, and interoperability with appropriate Boolean operators (ie, “OR,” “AND”), as shown in Table 2. The process was documented in accordance with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist, ensuring a rigorous and transparent review process (Checklist 1).

Eligibility Criteria

To keep the review focused, we applied predefined inclusion criteria during the title and abstract screening and full-text review phases. The criteria addressed article type, language, publication time frame, geographic focus, topical relevance to data modernization, public health focus, and outcome evaluation. These are summarized in Table 1.

Screening Process

Articles identified from database searches and additional sources were reviewed using the cloud-based platform, Rayyan QCRI [16]. Following the removal of duplicate and inaccessible records (n=228), a total of 237 records proceeded to title and abstract screening. Two reviewer writers (ZZ and SR) independently screened these articles, labeled as “include,” “exclude,” or “may be with blinding enabled.” When there was a disagreement between reviewers, a third author was consulted to make the final decision. For articles not accessible through institutional holdings, the authors contacted the source author or journal to obtain the article. Finally, 56 articles were selected for subsequent review of the full article [17]. During this phase again, reviewers (ZZ and SR) independently assessed each of the full articles and discussed any uncertainties related to study selection [18].

Data Extraction

The data were combined into a single spreadsheet in Microsoft Excel 2016 for checking and coding. The extraction form included the following categories: study objectives, manifestations of best practices, population of interest, framework/approach/action plan challenges and gaps in data modernization, proposed solutions, and approaches to addressing health inequities. To ensure clarity and comprehensiveness, the form was piloted on a sample of five articles by the research team. Data extraction for the remaining studies was conducted independently by two reviewers (ZZ and SR). The completed extraction tables were then reviewed collaboratively by both researchers. This process was iterative, allowing for refinement and ensuring that all relevant information necessary for thematic development was captured accurately.

Quality Assessment, Collating, Summarizing, and Reporting Results

In a stepwise process, findings were reported following the PRISMA-ScR guidelines. This PRISMA-ScR flow diagram details the study selection process, number of studies screened, and those assessed for eligibility, as well as with reasons for inclusions and exclusions at each stage [19,20]. The collaborative review process among coauthors further enhanced the robustness of the results’ interpretation, ensuring a comprehensive and well-rounded analysis of the gathered evidence. Finally, the following information was extracted from each of the articles: author(s) name, year of publication, research objectives or question(s), method, and summary of findings of the theoretical framework about best practice, data modernization challenges, and gaps. From the included studies, we extracted and analyzed key information to identify emerging themes and categories related to public health data modernization. A general analysis was then conducted to classify the studies based on technical focus areas and public health domains. Particular attention was given to identifying commonly used best practices, which were synthesized and highlighted across the included literature. One of the central aims of this review was to extract and categorize the implementation challenges reported by researchers. These challenges were analyzed to provide insight into barriers that future initiatives may encounter. The extracted data for each thematic category, including best practices and challenges, were systematically organized and presented using summary tables.

Results

Overview

This scoping review resulted in 21 papers on data modernization in US public health system and identified several key findings (Figure 2). Descriptive findings for each article are shown in Table 3. Seven studies utilized case studies or descriptive designs, primarily employing surveys, face-to-face interviews, and observations to collect data. Six studies applied mixed methods, integrating focus group discussions and interviews for a comprehensive perspective. Three studies used quantitative methods [21-23]. The remaining studies utilized a qualitative Delphi survey for gathering expert opinions and employed semistructured interviews to gather expert insights [24,25] (Table 4).

‎

Implementation Strategies and Best Practices

The review shows that modernizing public health data systems involves diverse strategies and best practices, which are categorized into 5 key areas: data enhancement, stakeholder engagement, surveillance and monitoring, data governance, and intervention and implementation, each emphasizing specific practices in public health, as shown in Tables 3 and 5.

Data Enhancement

Data enhancement involves improving the quality, completeness, and accessibility of data, which is essential for effective public health decision-making. Carney et al [7] and Heacock et al [38] emphasize building infrastructure to support data sharing, while Feldman et al [26] highlight enhancement efforts in Maryland’s COVID-19 contact tracing system through enriched electronic laboratory reports incorporating demographic, vaccination, and hospitalization data—an initiative led by the Maryland Department of Health in collaboration with Chesapeake Regional Information System for Our Patients. Additionally, Romero et al [36] focus on public health analytics for COVID-19, and Dong et al [34] explore strategies for data integration and real-time sharing to support timely responses and informed decision-making. Consequently, Allen et al [23] highlighted data enhancement through standardized electronic health records (EHR) data extraction, open-source integration tools, and governance policies to improve chronic disease surveillance and address health disparities (Table 5).

Stakeholder Engagement

Dore et al [9] and Brody et al [25] highlight the importance of stakeholder collaboration in enhancing public health data systems. Dore et al [9] focus on integrating EHR with Medicare claims through partnerships among providers, researchers, and agencies, while Bordy et al [25] emphasize the stakeholders’ role in improving data-sharing infrastructure for better pandemic response.

Surveillance and Monitoring

Surveillance and monitoring systems described in these studies emphasize the use of EHR data and real-time tools to enhance public health efforts. For instance, Cocoros et al [29] explain how the RiskScape platform utilizes EHR data for near real-time chronic disease monitoring, enabling timely insights into population health trends. Similarly, Hohman et al [30] highlight the Multi-State EHR-Based Network for Disease Surveillance, which integrates EHR data from health exchanges to provide geospatial analysis and state-level disease tracking. Additionally, Allen et al [23] demonstrate the use of EHR-derived data to improve hypertension prevalence estimates, offering more precise local insights compared to traditional survey methods. Finally, Dong et al [34] describe how Johns Hopkins transitioned from Google spreadsheets to a real-time COVID-19 dashboard, using fusion logic to efficiently manage global pandemic data and improve decision-making (Table 5).

Data Governance

Data governance in public health refers to a comprehensive framework of policies, roles, responsibilities, processes, and standards that determine who can access and use specific data, under what conditions, and for what purposes. It ensures data privacy, security, accountability, and transparency across systems and stakeholders. Standardization and protocols ensure consistency and interoperability across different public health data systems. According to Hohman et al [30], EHRs have been used to collect data from multiple health systems for real-time chronic disease surveillance. These protocols facilitated data integration, quality assessment, and visualization across varied formats, ensuring consistent data access and secure sharing. It applied statistical and geospatial methods to monitor and analyze chronic disease prevalence and trends across different populations. Related studies have discussed the implementation of standardized protocols, though these require proper referencing [30,31,38]. These studies reinforce the need for governance frameworks that adapt to technological advances while protecting individual privacy and enhancing data utility for public health action (Table 5).

Intervention and Implementation

Intervention and implementation focuses on the practical application of data modernization strategies to improve public health outcomes. This category includes studies that discuss intervention strategies but often lack detailed reporting on their long-term outcomes and effectiveness, highlighting the need for more comprehensive evaluations (Table 5) [28,35].

Data Modernization Challenges

As public health organizations work to modernize their data systems, they face a range of challenges that go beyond the technical issues that are summarized in Table 5. These challenges affect how data are collected, managed, and used, which in turn impacts the success of public health efforts. To build a strong and flexible public health data system that can handle both everyday needs and emergencies, it is important to understand these obstacles (Tables 3 and 6).

Sociodemographic Data Gaps

Gaps were noted in outdated data collection methods, insufficient workforce training, and fragmented governance structures. Sociodemographic data gaps present significant challenges in public health data modernization, particularly in accurately assessing disparities across different populations, which is reflected in other papers [7,22,28,29,35]. According to Dobrinen et al [22], the absence of detailed sociodemographic data prevents the Rocky Mountain Tribal Epidemiology Center from comprehensive monitoring and exacerbates health disparities during crises like COVID-19. Meanwhile, the study discusses the importance of establishing a nationwide health data ecosystem. One of the key strengths is its clear call for integration across state, local, and federal levels for improved public health data management, particularly in pandemic situations like COVID-19 [28]. Carney et al [7] highlight the lack of comprehensive sociodemographic data, particularly in identifying health disparities and a balance between data sharing and protecting privacy (Table 6).

Technical and Operational Challenges

Technical and operational challenges are pervasive in the modernization of public health data systems. Dignam et al [21] highlighted the complexities involved in ensuring continuous, accurate data reporting and the challenges of implementing sophisticated statistical models for public health surveillance. Similarly, Bambekova et al [32] addressed technical challenges in integrating social determinants of health data with clinical records. López-Martínez et al [33] further underscored the operational difficulties associated with processing massive volumes of data in real time, which is essential for predicting clinical complications before they occur. The need for automation and operations in big data analytics platforms also presents significant technical hurdles (Table 6) [21,22,30].

Gaps in Integration and Interoperability

Integration and interoperability of data from various sources are vital for a cohesive public health response. Feldman et al [26] addressed the challenges of integrating data across different platforms, ensuring data completeness and timeliness, which were crucial for the effectiveness of COVID-19 contact tracing. Hohman et al [30] further pointed out the technical challenges related to data extraction and transformation from various sources, emphasizing the need for consistent data governance and the protection of privacy. Similarly, aligning and integrating diverse datasets is particularly difficult when they originate from varying standards and formats (Table 6) [26,28,29].

Barriers in Data Governance and Privacy

Data governance and privacy remain critical issues as public health systems integrate more data sources. In response, data modernization initiatives require both effective management change and strategic technology implementation. For instance, Schmit et al [27] highlighted the limitations imposed by existing data use agreements, which restrict federal access to essential data during public health emergencies, thus hampering effective responses. Putnam-Hornstein et al [31] emphasized the ongoing need for training and support for public health staff to ensure they can effectively manage and interpret integrated data, addressing both governance and privacy concerns. Additionally, several studies identified the challenges posed by restrictive data systems and inconsistent regulations, which limit the interoperability between public health and health care systems, limiting their usefulness for timely decision-making (Table 6) [27,29,30].

Obstacle Between Data Quality and Completeness

The quality and completeness of data remain fundamental challenges in public health modernization efforts. Dore pointed out the necessity of a systematic EHR infrastructure, especially in long-term care settings, which had been previously neglected by initiatives such as the Health Information Technology for Economic and Clinical Health (HITECH) Act [9]. Similarly, Cocoros et al [29] emphasized the variability in data quality across different EHR systems, including issues with geographic coverage and the challenges of linking and deduplicating patient data from multiple care sites. These issues underline the importance of establishing robust systems to ensure comprehensive and high-quality data (Table 6) [9,21,27].

Discussion

Principal Findings

This scoping review identified a range of initiatives aimed at advancing public health data modernization in the United States, revealing both promising developments and persistent systemic challenges. While emerging efforts in areas such as EHR standardization, cross-sector collaboration, and real-time surveillance have shown potential [7,9,30,38], they remain limited in scale, inconsistently implemented, and often lack the infrastructure needed for long-term sustainability. In particular, barriers related to technical and operational capacity, integration and interoperability, data governance and privacy, and data quality and completeness continue to hinder progress across jurisdictions. The following discussion highlights modernized public health data systems that integrate advanced technologies and standardized practices, as well as the need for policy information to enhance data sharing and integration in public health systems. The COVID-19 pandemic has significantly accelerated data modernization efforts, presenting a unique opportunity to improve public health infrastructure. The primary challenge now is to sustain this thrust, ensuring efforts remain focused on the overarching goal of improving population health. In this context, the CDC’s strategic priorities, particularly accelerating data into action and managing change and governance, play a vital role [39]. Adopting this system-level perspective will be essential for achieving long-term success and maximizing the benefits of modernization initiatives [2,40].

These challenges faced in progressing data modernization are further elaborated in other studies. Across studies, recurring issues such as inconsistent data standards, fragmented infrastructures, and limited real-time access illustrate how structural weaknesses hinder effective and timely responses, particularly in tribal public health settings [22]. Moreover, we recommend standardized data integration and advanced linkage techniques to enhance the understanding of client needs and service utilization, thereby addressing health disparities [31]. Furthermore, the study underscores the pivotal role of big data and machine learning in strengthening public health interventions and decision-making, ultimately advancing health equity [33].

Although we did not find significant divergent views, the general perception in these reviewed studies highlights significant advancements and barriers in data modernization, emphasizing the importance of effective solutions and their impact on health equity. For instance, one of the studies revealed that collaboration between state health departments and health information exchanges to improve COVID-19 contact tracing served as a blueprint for future public health data modernization [26]. Another initiative focused on the development of a long-term care data cooperative to enhance care and research in nursing homes [9]. While not directly focused on health equity, this initiative supports an often underrepresented population in clinical studies [9].

Efforts to advance public health data modernization must build upon prior experiences in health care digitization. The HITECH Act, which catalyzed the widespread adoption of certified EHR, serves as a foundational example. While it successfully increased digital capacity, it also introduced new obstacles, including fragmented governance, limited interoperability, and system complexity that continue to hinder the seamless exchange of data across public health agencies [41].

A key insight from this review is the need to reform data use agreements. Existing agreements often limit data flow due to rigid language, security concerns, and a lack of adaptability across jurisdictions. Revising these agreements to support secure, timely, and flexible data sharing is essential for enabling rapid and coordinated public health responses, particularly in times of crisis [27]. Similar difficulties have been noted in other fields where the integration of data is essential. To limit its scientific and policy relevance, Kosnik et al [42] showed that integrated datasets may produce fragmented or inconsistent evidence if standardized reference and systematic validation are not used.

In addition to governance and legal frameworks, several studies emphasized the role of targeted surveillance tools in addressing environmental and health equity concerns. For instance, enhanced surveillance strategies for childhood lead exposure were shown to improve detection and response efforts in marginalized communities, where environmental risks are often concentrated [21]. These tools demonstrate how modernization can directly impact equity when tailored to specific population needs.

Standardization also emerged as a recurring theme in the literature. The adoption of Clinical Data Interchange Standards Consortium guidelines for real-world data was identified to ensure consistent and representative analysis, particularly for populations historically underrepresented in clinical trials. By standardizing data collection and analysis processes, these frameworks help mitigate biases and improve the inclusivity of health research [24].

Taken together, these findings reinforce the urgency of investing in core data infrastructure improvements. Achieving true interoperability across agencies, systems, and data sources requires transitioning to cloud-based platforms that support real-time automation, secure data access, and analytics on a scale. Without these foundational upgrades, the goals of equity, timeliness, and efficiency in public health modernization may remain out of reach [39,43,44].

From our scoping review, we identified that the challenge for public health is the decentralization of governance across 50 states [41]. These governance challenges are further compounded by shortcomings in demographic data collection, which restrict timely emergency responses and undermine equity in health outcomes [35]. In response, several studies advocate for stronger federal coordination and targeted funding mechanisms to support a more integrated, equitable data infrastructure that can address disparities amplified by the COVID-19 pandemic [28]. The literature also focuses on addressing structural racism in health data systems, proposing principles for equitable health data practices [3]. In parallel, targeted investments to strengthen analytic capacity in community health centers are essential, as timely and accurate data are critical for managing care among high-risk populations [36]. In response to these burning issues, public health policymakers must design a strategic approach that acknowledges the diverse starting points and capabilities across the nation, ensuring an inclusive and adaptable framework for data modernization best practices [41,43]. For this, now is the time for collaboration across jurisdictions to address common data challenges, with smaller or under-resourced health departments benefiting most from capacity building. Public health agencies should convene informatics staff to develop interoperable disease-agnostic systems that use common standards and technologies for efficient data integration.

The studies collectively underscore the need for targeted, equitable data modernization efforts to enhance public health interventions and address disparities. By implementing best practices such as standardized data models, secure data-sharing agreements, and advanced surveillance methods, public health agencies can significantly improve health outcomes and address inequities across various populations.

Limitations

This study has some limitations. We limited our search to four databases—PubMed, Scopus, CINAHL, and PsycINFO—along with Google Scholar. This may have excluded relevant studies found in other databases. Many of the studies related to public health and data modernization were qualitative or uncategorized, which prevented us from conducting a meta-analysis. Additionally, we focused only on US-based articles, which may have excluded valuable insights from international studies. Lastly, only English-language papers were included, potentially missing important perspectives from non-English research.

Conclusions and Recommendations

This scoping review highlights critical messages and interlinked issues in public health data modernization, including the impact on the modern data modernization initiative of the HITECH Act, the importance of adopting advanced technologies like North Star Cloud and Health Level Seven standards, and the ongoing challenges faced in the United States [11,12,45]. Studies reveal persistent issues with data visualization, reporting standardization, and real-time epidemiological data quality, essential for effective public health surveillance and response [40]. Despite advancements, we found that substantial gaps remain in population health informatics and informatics skills within the workforce in the field of data modernization. Governance issues, such as fragmented regulations and insufficient funding, complicate the development of a unified health data ecosystem. Misaligned reporting requirements, the absence of unique patient identifiers, and data gaps further hinder progress. Addressing these challenges requires robust policies and funding strategies to support standardized data collection, sharing, and utilization. Investing in data modernization, especially in informatics training and skill development, is crucial to fostering a more efficient, equitable, and responsive public health infrastructure.