Digital Health Data Quality Issues: Systematic Review

Background: The promise of digital health is principally dependent on the ability to electronically capture data which can be analysed to improve decision making. Yet, the ability to effectively harness data has proven elusive, which has largely been due to the quality of data captured. Despite the importance of data quality (DQ), an agreed upon DQ taxonomy evades literature. When consolidated frameworks are developed, the dimensions are often fragmented, without consideration of the interrelationships between the dimensions or their resultant impact. Objective: The aim of this study was to develop a consolidated digital health DQ dimensions and outcomes framework, which provided insights into the three research questions: 1) What are the dimensions of digital health DQ? 2) How are the dimensions of digital health DQ related? And 3) What are the impacts of digital health DQ? Methods: Following PRISMA guidelines, a developmental systematic literature review was conducted of peer-reviewed literature focussing on digital health DQ in predominately hospital settings. A total of 227 relevant articles were retrieved that were inductively analysed to identify digital health DQ dimensions and outcomes. The articles were inductively analysed, using open coding, constant comparison, and card-sorting with subject matter experts to identify the digital health DQ dimensions and digital health DQ outcomes. Subsequently, computer-assisted analysis was performed and verified by DQ experts to identify: the interrelationships between the DQ dimensions; and, relationships between DQ dimensions and outcomes. The analysis resulted in the development of the DQ dimensions and outcomes (DQ-DO) framework. Results: The digital health DQ-DO framework consists of 1) six dimensions of DQ: accessibility, accuracy, completeness, consistency, contextual validity, and currency; 2) interrelationships amongst the dimensions of digital health DQ, with consistency being the most influential dimensions impacting all other digital health DQ dimensions; 3) Five digital health DQ outcomes: clinical, clinician, research-related, business processes, and organizational outcomes; and 4) relationships between the digital health DQ dimensions and DQ outcomes; with the consistency and accessibility dimensions impacting all DQ outcomes. Conclusions: The DQ-DO framework developed in this study demonstrates the complexity of digital health data quality and the necessity for reducing digital health data quality issues. The framework further provides healthcare executives with holistic insights into DQ issues and resultant outcomes, which can help them prioritise which DQ-related problems to tackle.


Introduction Background
The healthcare landscape is changing globally owing to substantial investments in health information systems which seek to improve healthcare outcomes [1].Despite the rapid adoption of health information systems [2] and the perception of digital health as The papers returned from the search were narrowed down in a four-step process (Figure 1Error: Reference source not found).In the identification step 5177 studies were identified through multiple database searches with 3856 duplicates removed resulting in 1321 articles.The 1321 articles were randomly divided into six batches, which were assigned to separate researchers who applied the inclusion and exclusion criteria (Table 1).As a result of abstract screening 896 articles were excluded, resulting in 425 articles remaining.Following a similar approach to the abstract screening, the 425 articles were again randomly divided into six batches and assigned to one of six researchers to read and assess the relevance of the article in line with the selection criteria.The assessment of each of the 425 articles was then verified by the research team resulting in the final set of 227 relevant articles.During this screening phase (i.e., abstract and full-text), daily meetings were held with the research team with any uncertainties raised and discussed until consensus was reached by the team as to whether the article should be included or excluded from the search.In line with Templier and Paré [25], as this systematic literature review was developmental in nature rather than an aggregative meta-analysis, quality appraisals were not performed on the individual articles.

Table 1. Inclusion and exclusion criteria
Inclusion Exclusion Criteria Specifically focuses on data quality in digital health.
Development of algorithms for advanced analytics techniques (e.g., machine learning, artificial intelligence) without application within hospital settings.Empirical papers or review articles where conceptual frameworks were either developed or assessed.
Descriptive papers without a conceptual framework or empirical analysis.Considers digital health within hospital settings.
Focused only on primary care (e.g., general practice) Published in peer reviewed outlets within any timeframe Pre go-live considerations (e.g., software development) Published in English Theses and non-peer reviewed (e.g., white papers, editorials).

Literature Analysis
The relevant articles were imported in NVivo (v.12) where analysis was iteratively performed.To ensure reliability and consistency in coding, a coding rule book [26] was developed and progressively updated to guide the coding process.The analysis process involved six steps (Figure 2).
In the first step of the analysis, the research team performed open coding [26] where relevant statements from each article were extracted using verbatim codes while allowing initial grouping of similar concepts [27].The first round of coding resulted in 1298 open codes.Second, the open codes were segmented into two high level themes, the first group contained 1044 open codes pertaining directly to DQ dimensions (e.g., data accuracy); the second group contained 254 open codes related to DQ outcomes (e.g., financial outcomes).
In the third step, through constant comparison [28] the 1044 raw DQ codes were combined into 29 DQ sub-themes based on commonalities (e.g., contextual DQ, fitness for use, granularity, relevancy, accessibility, availability).In the fourth step, again through performing iterative and multiple rounds of constant comparison, the 254 open codes related to DQ outcomes were used to construct 22 initial DQ outcome sub-themes (e.g., patient safety, clinician-patient relationship, continuity of care).The DQ outcomes sub-themes were further compared to each other resulting in 5 DQ outcome dimensions (e.g., clinical, business process, research-related, clinician, and organisational).For the DQ sub-themes, constant comparison was performed facilitated by the card sorting method [29] where an expert panel of 8 DQ researchers formed into four groups assessed the sub-themes for commonalities and differences.The expert groups presented their categorisation to each other until a consensus was reached.This resulted in a consolidated set of six DQ dimensions (accuracy, consistency, completeness, contextual validity, accessibility, and currency).Appendix 2 provides an example of how the open codes, were reflected in sub-themes, and themes.
After identifying the DQ dimensions and outcomes, the next stage of coding progressed to identifying the interrelationships (Step 5) between the DQ dimensions as well as the relationships (Step 6) between the DQ dimensions and DQ outcomes.To do so, the matrix coding query function using relevant Boolean operators (AND, Near) in NVivo was performed.The outcomes of the matrix queries were reviewed and verified by an expert researcher in the health domain.
Throughout the analysis, steps were performed to provide credibility into our findings.Firstly, prior to commencing the analysis, the research team members who were extracting the verbatim codes initially independently reviewed three common articles, then convened to review any variations in coding.In addition, they reconvened multiple times a week to discuss their coding and update the codebook to ensure a consistent approach was followed.Coder corroboration was performed throughout the analysis with two experienced researchers independently verifying all verbatim nodes until consensus was reached [26].Subsequent coder corroboration was performed by two experienced researchers to ensure the open codes accurately mapped to the themes and the dimensions.This served to provide internal reliability.Steps were also performed to improve external reliability [107].Namely, the card-sorting method provided an expert appraisal.In addition, the findings were presented to and confirmed by three digital healthcare professionals.

Results
The vast majority of relevant articles were published in journal outlets (n=169), followed by conference proceedings (n=42), and book sections (n=16).The 169 journal articles were published in 107 journals, with 12% of the journals publishing more than one study (illustrated in Figure 3).The complete breakdown of how many articles have been published within each outlet is detailed in Appendix 3.
Step 6: Nvivo assisted relationships identification Step 5: NVivo assisted interrelationship identification Step 4: Constant comparison within subtheme (supported by card sorting) Step 3: Constant comparison within topic Step 2: Segment codes into relevant themes Step

Data Quality Definitions
Multiple definitions of DQ are discussed in the literature (Appendix 4).There is no consensus on a single definition of DQ, however analysis of the definitions reveals two perspectives, which we label as the 1) context-agnostic perspective, and 2) contextaware perspective.The context-agnostic perspective defines DQ based on a set of dimensions regardless of the context within which the data is used.For instance, as [30] notes "documentation and contents of data within an electronic medical record (EMR) must be accurate, complete, concise, consistent and universally understood by users of the data, and must support the legal business record of the organization by maintaining the required parameters such as consistency, completeness and accuracy."Conversely the context-aware perspective evaluates the dimensions of DQ with recognition of the context within which the data is being used.For instance, as [31,32] notes DQ is "the degree to which data satisfy the requirements defined by the product-owner organization", and can be reflected through its dimensions such as completeness and accuracy.

Data Quality Dimension: Accessibility
The accessibility dimension (n=28, 12.3%) is composed of both the accessibility (n=15) and availability sub-themes reflecting the feasibility for users to extract data of interest [18].Scholars regularly view the accessibility sub-theme favourably with the increased adoption of electronic health record systems (EHRs) overcoming physical and chronological boundaries associated with paper records by allowing access to information from multiple locations at any time [36,37].Top et al. [36] notes that EHR made it possible for nurses to access patient data, resulting in improved decision making.Rosenlund et al. [38] further notes that EHRs benefit healthcare professionals through providing increased opportunities for searching and utilising information.The availability sub-theme is an extension of the accessibility sub-theme and examines whether the data exists and whether the data is in a format that is readily usable [39].
For instance, Dentler et al. [39] notes that pathology reports although accessible are recorded in a non-structured, free-text format making it challenging to readily use the data.While structured data may make data more available, Yoo et al. [40] highlights that structured data entry in the form of drop-down lists and check boxes tend to reduce the narrative description of patients' medical conditions.While not explicitly investigating accessibility, Makeleni and Cilliers [33] also note the challenges associated with structured data entry.
Studies often referred to accuracy as the 'correctness' of data, which is the degree to which data correctly communicates the parameter being represented [35].Conversely, others focus on plausibility, the extent to which data points are believable [42].While accuracy concerns were present for all forms of digital health data, some studies focused specifically on inaccuracies with diagnostic data, where "the accurate and precise assignment of structured [diagnostic] data within EHRs is crucial" [43], which is "key to supporting secondary clinical data" [44].
To assess accuracy, the literature regularly asserts that data needs to be validated against metadata constraints, system assumptions, and local knowledge [19] and conform to structural and syntactical rules.According to Kahn et al. [19], Sirgo et al. [45], conformance focusses on compliance of data with internal or external formatting, relational, or computational definitions.Accurate, verified, and validated data, as well as data conforming to standards contributes to integrity of data.Integrity requires that the data stored in health information systems is accurate and consistent, where the "improper use of [health information systems] can jeopardise the integrity of a patient's information" [33].An emerging sub-theme of accuracy was the veracity of data, which represents uncertainty in the data due to inconsistency, ambiguity, latency, deception, and model approximations [21].It is particularly important in the context of the secondary use of big data, where "data veracity issues can arise from attempts to preserve privacy, …and is a function of how many sources contributed to the data."[46] Data Quality Dimension: Completeness The completeness dimension (n=114, 50%) is composed of six sub-themes: missing data (n=66), level of completeness (n=25), representativeness (n=13), fragmentation (n=8), and breadth of documentation (n=2).A well-accepted definition of data completeness considers four perspectives: documentation (the presence of observations regarding a patient in data), breadth (the presence of all desired forms of data), density (the presence of a desired frequency of data values over time), and predictive (the presence of sufficient data to predict an outcome) [47].Our analysis revealed that these four perspectives, while accepted, are rarely systematically examined in extant literature, rather papers tend to discuss completeness or the lack thereof as a whole.
Missing data is a prominent sub-theme and represents a common problem in EHR data.
For instance, Gloyd et al. [48] argue that incomplete, missing and implausible data "was by far the most common challenge encountered".Scholars regularly identified that data fragmentation contributed to incompleteness, with a patient's medical record deemed incomplete due to data being required from multiple systems and EHRs [18,[49][50][51][52][53][54][55]."Data were also considered hidden within portals, outside systems, or multiple EHRs, frustrating efforts to assemble a complete clinical picture of the patient" [50].More positive perspectives pertaining to data completeness focus on the level of completeness, with studies reporting relatively high completeness rates in health datasets [37,39,[56][57][58][59].For data to be considered complete it needs to be captured at sufficient breadth and depth over time [12,18].Some studies have proposed techniques to improve completeness, which include: developing fit-for-purpose user interfaces [60][61][62], standardizing documentation practices, [63,64], automating documentation [65], and performing quality control [64].
In some instances, the level of completeness and extent of missing data differed depending on the nature of the patient [15,16,18,20,46,51,59,[66][67][68][69][70][71][72], which we classified into the sub-theme of representativeness.It has been found that there is "a statistically significant relationship between EHR completeness and patient health status" [70] with more data recorded for sick patients compared to less acute patients.This aligns strongly with the sub-theme of contextual validity.
Inconsistent data capturing is a prevalent sub-theme caused by the manual nature of data entry in healthcare settings [46], especially when data involves multiple times, teams, and goals [73].Inconsistent data capturing results in data variability and temporal variability.Data variability refers to inconsistency in the data captured within and between health information systems, whereas temporal variability reflects inconsistencies that occur over time and may be due to changes to policies or medical guidelines [20,48,[74][75][76][77][78][79].Semantic inconsistency (i.e., data with logical contradictions) and representational inconsistency (i.e., data variations due to multiple formats) can also result from inconsistent data capturing [80].
Standardization in terms of terminology, diagnostic codes, and workflows [64] are proffered to minimise inconsistency in data entry, yet in practice there is a "lack of standardized data and terminology" [9] and "even with a set standard in place not all staff accept and follow the routine" [64].The lack of standardisation is further manifested due to health information system differences across settings [81].As a result of the differences between systems, concordance -the extent of "agreement between elements in the EHR, or between the EHR and another data source" is hampered [82].
Inconsistent data entry can be further caused by redundancy within the system due to structured versus unstructured data [83], which we label as the sub-theme 'structuredness' and duplication across systems [66,78,[84][85][86][87], which we label as the sub-theme 'uniqueness'.While structured data entry, "facilitates information retrieval" [36] and is "in a format that enables reliable extraction" [18], the presence of unstructured fields leads to data duplication efforts, hampering uniqueness as data is recorded in multiple places with varying degrees of granularity and level of detail.
Contextual DQ is often described as 'fitness of use' [20] for which understanding the context in which data is collected is deemed important [18,51].Another factor that contributes to data being fit for use is granularity of data.Adequate granularity of timestamps [89], patient information [16], and data present in EHR (e.g., diagnostic code [16]) was considered important to make data fit for use.Finally, for data to be fit for use it needs to be relevant.As indicated by Schneeweiss and Glynn [69], for data to be meaningful healthcare databases need to contain relevant information of sufficient quality, which can help answer specific questions.The literature clearly demonstrates the need to take context into consideration when analysing data and the need to adapt technologies to the healthcare context so that appropriate data is collected for reliable analysis to be performed.

Data Quality Dimension: Currency
The currency dimension (n=18, 8%) was formed by the single sub-theme of timeliness.Currency or timeliness, is defined in Afshar et al. [35] and Makeleni and Cilliers [33] as the degree to which data represents reality from the required point in time.From an EHR perspective, the data should be up to date, available, and reflect the profile of the patient at the time the data is accessed [35,90].Lee et al. [42] extends this to include the recording of an event at the time it occurs such that a value is deemed current if it is representative of the clinically relevant time of the event.Frequently mentioned causes for lack of currency of data include: (i) recording of events (long) after the event actually occurred [52,64,91,92], (ii) incomplete recording of patient characteristics over time [16], (iii) system/interface design not matching workflow and impeding timely recording of data [64], (iv) mixed mode recording -paper and electronic [64], and (v) lack of timestamp metadata meaning the temporal sequence of events is not reflected in the recorded data [16].

Interrelationships between the Data Quality Dimensions
As illustrated in Figure 5 and evidenced in Appendix 6, interrelationships were found between the digital health DQ dimensions.

Figure 5. Interrelationships between DQ Dimensions
Consistency influenced all DQ dimensions.Commonly these relationships were expressed in terms of the presence of structured and consistent data entry prompting complete and accurate data to be entered into the health information system, which provides more readily accessible and current data for healthcare professionals when treating patients.As Roukema et al. [37] notes "structured data entry applications can prompt for completeness, provide greater accuracy and better ordering for searching and retrieval, and permit validity checks for DQ monitoring, research, and especially decision support".When data is entered inconsistently it impedes the accuracy of the medical record and the contextual validity for secondary uses of data [67].
Accessibility of data was found to influence the currency dimension of DQ.When data is not readily accessible it seldom satisfies the timeliness of information for healthcare or research purposes [39].Currency also influenced the accuracy of data.In a study investigating where DQ issues in EHR arise , it was found that "false negatives and false positives in the problem list sometimes arose when the problem list … [was] out-ofdate, either because a resolved problem was not removed or because an active problem was not added" [51].
Completeness further influenced the accuracy of data as [33] notes "data should be complete to ensure it is accurate".The presence of inaccurate data was regularly linked to information fragmentation [49], incomplete data entry [86], and omissions [42].
Completeness also influenced contextual validity as it is necessary to have all the data available to complete specific tasks [32].When it comes to the secondary use of EHR data, evaluation of "completeness becomes extrinsic, and is dependent upon whether or not there are sufficient types and quantities of data to perform a research task of interest" [70].
Accuracy and contextual validity exhibited a bidirectional relationship with each other.
The literature suggests that accuracy influences contextual validity, however data cannot simply be extracted from structured form fields, free text fields will also need to be consulted.For instance, Kim and Kim [93] identifies "it is sometimes thought that structured data are more completely optimized for clinical research.However, this is not always the case, particularly given that extracted EMR data can still be unstable and contain serious errors."Conversely, other literature suggests that when only a segment of information regarding a specific clinical event (i.e., contextual validity) is captured inaccuracy can result [16].

Outcomes of Digital Health Data Quality
The analysis of literature identified five types of digital health DQ outcomes: 1) clinical, 2) business process, 3) clinician, 4) research related, and 5) organisational outcomes (Appendix 7).Through utilising NVivo's built-in crosstab query coupled with subject matter expert analysis, it was identified that different DQ dimensions were related to DQ outcomes in different ways (Table 3).Currency was the only dimension that did not have a direct effect on DQ outcomes.However, as discussed later (Figure 6), it is plausible that currency affects DQ outcomes through impacting other DQ dimensions.Below, we discuss each DQ dimension and their respective outcomes.
*Note: X denotes relationship between DQ dimension and outcome is reported in literature.Blank cells denote that there is no evidence to support the relationship.
We identified that the accessibility DQ dimension influenced clinical, clinician, business process, research-related, and organisational outcomes.In terms of clinical outcomes, Roukema et al. [37] indicates that EHRs through improving accessibility and legibility of healthcare data significantly enhances the quality of patient care.The increased accessibility of medical records during the delivery of patient care is further proffered to benefit clinicians through reducing data entry burden [36].Conversely, inconsistency in the availability of data across health settings increases clinician workload, as Wiebe et al. [15] notes "given the predominantly electronic form of communication between hospitals and general practitioners in Alberta, the inconsistency in availability of documentation in one single location can delay processes for practitioners searching for important health information".When data is accessible and available it can improve business processes (e.g., quality assurance) and research-related (e.g., outcomesoriented research) outcomes and is able to support organisational outcomes with improved billing and financial management [94].
The literature demonstrates that data accuracy influences clinical outcomes [14,66,95] and research-related outcomes [14,96], as Wang et al. [14] describes, "errors in healthcare data are numerous and impact secondary data use and potentially patient care and safety".Downey et al. [66] observe the negative impact on quality of care (i.e., clinical outcomes) resulting from incorrect data and state "manual data entry remains a primary mechanism for acquiring data in EHRs, and if the data is incorrect then the impact to patients and patient care could be significant" [66].Poor data accuracy also diminishes the quality of research outcomes.Precise data is beneficial in producing high quality research outcomes as Gibby [96] explains, "computerized clinical information systems have considerable advantages over paper recording of data, which should increase the likelihood of their use in outcomes research.Manual records are often inaccurate, biased, incomplete, and illegible".Closely related to accuracy, contextual validity is an important DQ dimension which considers the fitness for research as stated by Weiskopf et al. [70] "[w]hen repurposed for secondary use, however, the concept of "fitness for use" can be applied".
The consistency DQ dimension was related to all DQ outcomes.It was commonly reported that inconsistency in data negatively impacts the reusability of EHR data for research purposes hindering research-related outcomes and negatively impacting business processes and organizational outcomes.For example, Kim et al. [97] acknowledge that inconsistent data labelling in EHR systems may hinder accurate research results noting, "a system may use local terminology that allows unmanaged synonyms and abbreviations.… If local data are not mapped to terminologies, … performing multicentre research would require extensive labour".Alternatively, von Lucadou et al. [16] indicates the impact of inconsistency on clinical outcomes reporting that the existence of inconsistencies in captured data "could explain the varying number of diagnoses throughout the encounter history of some subjects".Whereas, Diaz-Garelli et al. [43] demonstrate the negative impact that inconsistency has on clinicians in terms of increased workload.
Incomplete EMR data was found to impact clinical outcomes (e.g., reduced quality of care), business process outcomes (e.g., interprofessional communication), researchrelated (e.g., research facilitation), and organizational outcomes (e.g., key performance indicators related to readmissions) and research related outcomes [15].For example, while reviewing the charts of 3011 non-obstetric inpatients, Wiebe et al. [15] found that missing discharge summary within an EHR "can present several issues for healthcare processes, including hindered communication between hospitals and general practitioners, heightened risk of readmissions, and poor usability of coded health data", among other widespread implications.Liu et al. [98] further reports that "having incomplete data on patients' records has posed the greatest threat to patient care".Due to the heterogenous nature (with multiple data points) of EHR data, Richesson et al. [20] emphasise that access to large, complete data will allow clinical investigators "to detect smaller clinical effects, identify and study rare disorders, and produce robust, generalisable results".

Discussion
The following sections describe the three main findings of this research: 1) identification of the dimensions of data quality, 2) the interrelationships between the dimensions of data quality, and 3) the outcomes of data quality.As described in the 'Summary of Key Findings' section, these three findings led to the development of the DQ Dimensions and Outcomes (DQ-DO) framework.Subsequently, we compare the DQ-DO framework with related work.This leads to the generation of implications for future research.The discussion concludes with a reflection of the limitations of this study.

Summary of key findings
In summary, we unearthed three core findings.Firstly, we identified six dimensions of DQ within the digital health domain: consistency, accessibility, completeness, accuracy, contextual validity, and currency.These dimensions were synthesised from 30 subthemes described in the literature.We found that consistency, completeness, accuracy are the predominant dimensions of DQ.Comparatively speaking, limited attention has been paid to the dimensions of accessibility, currency, and contextual validity.Secondly, we identified interrelationships between these six dimensions of digital health DQ (Table 2).The literature indicates that data inconsistencies can influence all other DQ dimensions.The accessibility of data was found to influence the currency of data.Completeness impacts accuracy and contextual validity, with these dimensions serving as dependent variables and exhibiting a bidirectional relationship with each other.Thirdly, we identified five types of data outcomes (Table 2, Appendix 7): research-related, organisational, business process, clinical, and clinician.Consistency was found to be a very influential dimension impacting all types of DQ outcomes.Contextual validity on the other hand, was shown to be particularly important for data reuse (e.g.performance measurement, outcome-oriented research etc.).Whilst currency does not directly impact any outcomes, it impacts the accuracy of data, which impacts clinical and research-related outcomes.Therefore, if currency is not resolved, accuracy issues would still prevail.If the objective is to improve organisational outcomes, consistency, accessibility, and completeness were shown to be important considerations.Through consolidating our three core findings, we developed a consolidated DQ Dimensions and Outcomes framework, DQ-DO (Figure 6).

Comparison to literature
Our findings extend previous studies on digital health DQ in three ways.Firstly, through our rigorous approach, we identified a comprehensive set of DQ dimensions, which both confirms and extends existing literature.For instance of Weiskopf and Weng [17] identified five DQ dimensions including completeness, correctness, concordance, plausibility, and currency, all of which are present within our DQ framework, although in some instances, we use slightly different terms (referring to correctness as accuracy and concordance as consistency).Extending the framework of Weiskopf and Weng [17], we view plausibility as a sub-theme of accuracy, disentangle accessibility from completeness, and we also stress the importance of contextual validity per Richesson et al. [20].Others have commonly had a narrower perspective of DQ focusing on completeness, correctness, and currency [18], or on completeness, timeliness, and accuracy [13].In other domains of digital health, such as physician-rating systems, Wang and Strong's [99] data quality dimensions of intrinsic, contextual, representational, and accessibility have been adopted.Such approaches to assessing data quality are appropriate although it removes a level of granularity that is necessary to understanding relationships and outcomes.This is particularly necessary given the salience of consistency in our data set and the important role it plays in generating outcomes.
Secondly, unlike previous studies on DQ dimensions, we also demonstrate how these dimensions are all related to each other.By analysing the interrelationships between these DQ dimensions, we can determine how a particular dimension influences another and in which direction this relationship is unfolding.This is an important implication for digital health practitioners as whilst several papers have examined how to validate [57] and resolve data quality issues [16], to resolve issues with a specific DQ dimension requires awareness of the interrelated DQ dimensions.For instance, to improve accuracy, one also needs to consider improving consistency and completeness.
Thirdly, although previous studies describe how DQ can impact a particular outcome (e.g., [18,100,101]), they largely focus broadly on data quality, or a specific dimension of data quality, or on a specific outcome.For instance, Sung et al. [102] notes that poor quality data were a prominent barrier hindering adoption of digital health systems.Conversely, Kohane et al. [103] focus on research-related outcomes in terms of publication potential and identified that incompleteness and inconsistency can serve as core impediments.To summarise, the DQ-DO framework (Figure 6) developed through this review provides not only the dimensions and the outcomes but also the interrelationships between these dimensions and how they influence outcomes.

Implication 1: Equal Consideration across Data Quality Dimensions
This study highlights the importance of each of the six DQ dimensions: consistency, accessibility, completeness, accuracy, contextual validity, and currency.These dimensions have received varying attention in the literature.Although we observe that some DQ dimensions such as accessibility, contextual validity, and currency are discussed less frequently than others, it does not mean that these dimensions are not important for assessment.This is evident in Figure 6, which identifies that all DQ dimensions except for currency directly influence DQ outcomes.Whilst we did not identify a direct relationship between the currency of data and the six types of data outcomes it is likely that the currency of data influences the accuracy of data, which subsequently influences the research-related and clinical outcomes.Future research, including consultation with a range of stakeholders, needs to further delve into understanding the under-researched DQ dimensions.For instance, both currency and accessibility of data are less frequently discussed dimensions in the literature yet, with the advances in digital health technologies, both have become highly relevant for realtime clinical decisions [21,104].

Implication 2: Empirical Investigations of the Impact of the Data Quality dimensions
The DQ-DO framework identified in this study has been developed through a rigorous systematic literature review process, which synthesised literature related to digital health DQ.To extend this study, we advocate for empirical mixed-methods case studies to validate the framework, including an examination of the interrelationships between DQ dimensions and DQ outcomes, based on real-life data and consultation with a variety of stakeholders.To identify the presence of issues with DQ dimensions within digital health system logs existing approaches could be used [57,105].The DQ-outcomes could be assessed by extracting pre-recorded key performance indicators from case hospitals and be triangulated with interview data to capture patients, clinicians, and hospitals perspectives of impacts of DQ.This could then be incorporated into a longitudinal study, where data collection is performed prior to and after a DQ improvement intervention being performed, which would provide efficacy to the digital health DQ intervention.

Implication 3: Understanding the Root Causes of Data Quality Challenges
Although this study provides a first step towards a more comprehensive understanding of DQ dimensions for digital health data and their influences on outcomes, it does not explore potential causes of such DQ challenges.Without understanding the reasons behind these DQ issues, the true potential of evidence-based healthcare decisionmaking remains unfulfilled.Future research should examine the root causes of DQ challenges in healthcare data with a view to prevent such errors from occurring in the first place.One framework that may prove useful to illuminating the root-causes of DQ is the Odigos framework, which indicates that DQ issues emanates from the social world (i.e., macro and situational structures, roles, and norms), material world (e.g., quality of the EHR system and technological infrastructure), and the personal world (e.g., characteristics and behaviours of healthcare professionals) [105].These insights could then be incorporated into a data governance roadmap for digital hospitals.

Implication 4: Systematic assessment and remedy of Data Quality Issues
Though prevention remains better than the cure (see previous limitation), not all DQ errors can be prevented or mitigated.It is common for many healthcare organisations to dedicate resources to data cleaning in order to obtain high quality data in a timely manner and this will remain necessary (though hopefully to a lesser degree).Some studies (e.g., [18]) advocate evidence-based guidelines and frameworks for a detailed assessment of the quality of digital health data.However, there is little work focusing on a systematic and automated way of assessing and remedying common DQ issues.Future research should also focus on evidence-based guidelines, best practices, and automated means to assess and remedy digital health data.

Limitations
This review is scoped to studying digital health data generated within a hospital setting and not to other healthcare settings.This is necessary because of the vast differences between acute health care settings and primary care.Future research should seek to investigate the digital health data of primary care settings to identify the DQ dimensions and outcomes relevant to these settings.In addition, this literature review has been scoped to peer-reviewed outlets, with "grey" literature excluded, which could have led to publication bias.Although this scoping may have missed some articles, it was necessary to ensure quality behind the development of the digital health DQ framework.An additional limitation that may be raised by our method is that due to the sheer amount of articles returned by our search, we did not perform double coding (where independent researchers analyse the same article).To mitigate this limitation, steps were taken to minimise bias through conducting coder corroboration sessions and group validation as mentioned in the Methods section with the objective of improving internal and external reliability [107].To further improve internal reliability two experienced researchers verified the entirety of the analysis in NVivo and for external reliability card sorting assessments were performed with data quality experts and the findings were presented and confirmed by three digital healthcare professionals.
Furthermore, empirical validation of the framework is required, both in terms of reallife data and input from a range of experts.

Conclusions
The multidisciplinary systematic literature review conducted in this study resulted in the development of a consolidated digital health DQ framework comprised of six DQ dimensions, the interrelationships between these dimensions, six DQ outcomes, and relationships between these dimensions and outcomes.We identified four core implications to motivate future research: specifically researchers should: 1) pay equal consideration to all dimensions of data quality as the dimensions can both directly and/or indirectly influence DQ outcomes; 2) seek to empirically assess the DQ-DO framework using a mixed-methods case study design;  The totality of features & characteristics of an entity that bears on its ability to satisfy stated and implied needs [109] Data's "fitness for use" and can be described by a set of dimensions (e.g., accuracy and completeness) [110] The ability of the data to fulfil the purpose for which they were collected or fit for use.The concept of 'fitness for use' emphasises the importance of taking the end user's perspective of quality into account because it is the end users who will decide whether a product is fit for use or is conforming to specific requirements [33] Data which is accurate, reliable, "fit for use" and relevant [111] Data fit for use, where fitness for use produces accurate, complete, and timely data accessible to stakeholders and relevant to their tasks [112] DQ is "fit-for-use" in that its determinants are dependent on the data consumer's expectations, in the context of a specific purpose for data use.[113] DQ is most commonly defined as 'fitness for use' [20] DQ: Context Agnostic Perspective Documentation and contents of data within an electronic medical record (EMR) must be accurate, complete, concise, consistent and universally understood by users of the data, and must support the legal business record of the organization by maintaining the required parameters such as consistency, completeness and accuracy.[30] EHR DQ dimensions: completeness, correctness, concordance, plausibility, and currency.[22] Relevant, necessary, accurate, complete, and updated data [114] Data that are accurate, relevant, valid, reliable, legible, complete, and available when it is needed by decision-makers for healthcare delivery and planning purposes; DQ consists of six primary dimensions, which includes completeness, consistency, conformity, accuracy, integrity and timeliness [33] Variations in expected data versus collected data (e.g., timeliness, accuracy) are collectively referred to as DQ [115] Three DQ categories: conformance, completeness, and plausibility [116] EMR DQ dimensions: correctness (i.e., accuracy), completeness, concordance (i.e., accessibility), currency (i.e., timeliness), and plausibility (i.e., relevancy). [17] The core framework includes three constructs of DQ: complete, correct, and current data… EHR data completeness can be defined in multiple ways, depending upon intended use, and that, in turn, efforts to calculate rates of records completeness would vary based upon these different definitions and uses [18] "accuracy, believability, reputation, objectivity, factuality, consistency, freedom from bias, correctness, and unambiguousness."[21] Three categories: currency, completeness, and correctness.To estimate correctness, two further categories-plausibility and concordance-were used [117] A proper assessment of DQ will examine the data from several perspectives or dimensions including validity, accuracy, completeness, relevance, timeliness, availability, comparability, consistency, duplication, integrity and conformity [118] Accuracy, availability, usability, integrity, consistency, standardisation and timeliness are some characteristics of high-quality data [62] Appendix 5: Evidence of the Sub-  The "timely and efficient access to all relevant information" [66] streamlines clinical practice and minimises unnecessary tasks [15,180,181]  The absence of a discharge summary can hinder communication between hospitals and general practitioners [15] Clinician The extent to which digital health DQ impacts frontline healthcare professionals.
 Nurses identified that EHR data will eliminate paperwork, improve ability to monitor patients, and decrease their workflow [36]  Poor data quality increases workload due to the documentation burden associated with inconsistent diagnosis codes [43] and inconsistency between data recorded across health settings [15]

Researchrelated
The extent to which the reusability of digital health DQ impacts clinical research outcomes.

Organisational
The extent to which digital health DQ impacts institutional finances, policy, and regulation compliance.
 "High DQ in medical records is fundamental to good clinical practice, program management and ultimately to policy decisions" [30] and further supports auditing and monitoring [30,67,81,183]  DQ issues can negatively impact institutional finances and regulatory compliance.[117]

Figure 4 :
Figure 4: Publications by year

Figure 6 .
Figure 6.Consolidated Digital Health Data Quality Dimensions and Outcomes (DQ-DO) Framework 1: Extraction of Open Codes Figure 3: List of top 12% journals Overall, as illustrated in Figure 4, the interest in digital health data quality has been increasing over time, with sporadic interest prior to 2006.

Table 3 .
The Relationships between DQ Dimensions and Data Outcomes 3) identify the root causes of the digital health DQ issues; 4) develop interventions to mitigate and prevent DQ issues from arising .The DQ-DO framework provides healthcare executives (e.g., chief information officers, chief clinical informatics officers) with insights into DQ issues, and which digital health-related outcomes they have an impact on -this can help them prioritise tackling DQ-related problems.None declared [33]]sonalised delivery of care and increased patient safety"[180] "When there is a gap or incomplete data from what is expected can lead to poor or delayed patient care that can lead to death, e.g., wrong results to wrong patient"[33] effective