This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
The benefits from the combination of 4 clinical information systems (CISs)—electronic health records (EHRs), health information exchange (HIE), personal health records (PHRs), and telehealth—in primary care depend on the configuration of their functional capabilities available to clinicians. However, our empirical knowledge of these configurations and their associated performance implications is very limited because they have mostly been studied in isolation.
This study aims to pursue 3 objectives: (1) characterize general practitioners (GPs) by uncovering the typical profiles of combinations of 4 major CIS capabilities, (2) identify physician and practice characteristics that predict cluster membership, and (3) assess the variation in the levels of performance associated with each configuration.
We used data from a survey of GPs conducted throughout the European Union (N=5793). First, 4 factors, that is, EHRs, HIE, PHRs, and Telehealth, were created. Second, a cluster analysis helps uncover clusters of GPs based on the 4 factors. Third, we compared the clusters according to five performance outcomes using an analysis of variance (ANOVA) and a Tamhane T2 post hoc test. Fourth, univariate and multivariate multinomial logistic regressions were used to identify predictors of the clusters. Finally, with a multivariate multinomial logistic regression, among the clusters, we compared performance in terms of the number of patients treated (3 levels) over the last 2 years.
We unveiled 3 clusters of GPs with different levels of CIS capability profiles:
Different CIS capability profiles may lead to similar equifinal performance outcomes. This underlines the importance of looking beyond the adoption of 1 CIS capability versus a cluster of capabilities when studying CISs. GPs in the strong cluster exhibit a comprehensive CIS capability profile and outperform the other two clusters with noncomprehensive profiles, leading to significantly high performance in terms of the quality of care provided to patients, efficiency of the practice, productivity of the practice, and improvement of working processes. Our findings indicate that medical practices should develop high capabilities in all 4 CISs if they have to maximize their performance outcomes because efforts to develop high capabilities selectively may only be in vain.
Over the past several years, a consensus has emerged on the recognition of the potential of clinical information systems (CISs) to improve the health care delivered to patients and save lives [
Electronic health records (EHRs) are at the heart of the reform of health systems in many developed countries [
Due to their potential benefits, previous decades have witnessed rapid growth in the adoption of EHRs in health care settings. However, despite considerable investments by governments, the adoption of EHRs in some primary care organizations has been slow, especially in small practices [
In addition to the abovementioned CISs (EHR and HIE), the personal health record (PHR) has recently been gaining attention because of its potential to support the transformation of health systems to a more patient-centered model of care [
Similarly, telehealth has been gaining attention because of its potential to reduce barriers to access health care and to save time and reduce costs for remote patients [
In conclusion, EHR, HIE, PHR, and telehealth can be considered the most important components of a modern and desirable CIS for both hospital and primary care practices. However, in previous research, these 4 CISs have been studied in isolation with little or no attention to their combination and associated implications for performance outcomes. A search in Medical Literature Analysis and Retrieval System Online (MEDLINE); March 2020) using the 4 terms, “
This paper takes a configurational perspective. In a broad sense, configuration is defined as “any multidimensional constellation of conceptually distinct characteristics that commonly occur together” [
Following Miller [
This study has 4 primary goals. The first is to identify predictors of the adoption of clinical systems by general practitioners (GPs) in Europe. The second is to identify and characterize GPs by uncovering typical profiles or patterns of the combination of 4 major CIS capabilities: EHRs, HIEs, PHRs, and telehealth. Consequently, this second objective is inductive in nature, empirically based, and taxonomic, dedicated to classification and subdivision [
We used a data set provided by the European Commission (EC) from the 2018 survey of European GPs. The objective of this study was to understand and measure the actual adoption and use of information and communication technology (ICT) and electronic health (eHealth) applications by general practitioners (GPs) in the 27 countries of the European Union (EU27) as well as changes in uptake over time. The 2018 survey of European GPs was a follow-up study of the 2013 survey, which included the EU27 plus 4 other countries (Croatia, Iceland, Norway, and Turkey).
Given that this 2018 survey of European GPs was a follow-up of the 2013 survey, eHealth is broadly defined, as in the previous study, as “the use of Information and Communication Technologies (ICT) across the whole range of health care functions” [
The questionnaire was composed of 3 parts: (1) GPs’ sociodemographics, organizational settings, practice location, description of tasks, and workload; (2) ICT availability and use within a GP practice that is divided into 4 categories: EHRs, HIE, telehealth, and PHRs; and (3) attitudinal questions as well as questions related to motivations, perceived barriers, and impacts of ICT.
Following the previous 2013 survey approach, a final sample of 5793 GPs was randomly selected over the analyzed EU27, with an overall sampling error of plus or minus 1.30% [
Given that the 4 objectives of this study are related to CIS adoption and associated with the implications for performance outcomes, out of the initial sample of 5793 GPs, only the 5244 who had an EHR system and stored patient data electronically were considered for a subsequent analysis. Due to the presence of missing values in 3 variables (HIE, PHR, and telehealth), we applied a multiple imputation strategy. Among the 5244 subjects, 5022 (95.77%) subjects had complete data, 100 (1.91%) subjects had missing values on telehealth only, 72 (1.37%) subjects had missing values on PHR only, and the remaining 50 (<1%) subjects had other missing patterns.
We imputed the missing values using a multiple imputation procedure. On the one hand, multiple imputation methods perform better than single imputation ones. We selected the multiple imputation method of fully conditional specification with the Proc MI procedure in SAS software version 9.4 (SAS Institute) because of its flexibility in allowing us to define the multivariate model by a series of conditional models, one for each incomplete variable [
On the other hand, it is impossible to use multiple imputed data sets for cluster analysis as it produces different results of clusters, and it seems that there are no methods to combine these results. Therefore, we used the multiple imputation method as discussed but only chose
Although the final 4 variables used for the cluster analysis were the ones with an imputation, we performed the same cluster analysis with the same variables without the imputation of missing values.
As stated earlier, the 549 GPs who did not store patients’ data electronically and those who did not have an EHR were excluded from the analysis. The final sample in our analysis was 5793 – 549 = 5244 subjects.
Characteristics of the respondents and their practices.
Variables, characteristics (ie, levels for categorical variables) | Nonsampled (n=549) | Sampled (n=5244) | Chi-square ( |
|||
|
|
|
|
|
|
|
|
Male | 247 (45.0) | 2652 (50.57) | 62 (1) | N/Aa | .01 |
|
Female | 302 (55.0) | 2592 (49.43) | 62 (1) | N/A | .01 |
Age (years), mean (SD) | 53.23 (10.35) | 51.86 (10.82) | N/A | 2.8 (5791) | .005 | |
Years spent in general practice, mean (SD) | 21.07 (11.13) | 20.83 (11.22) | N/A | 0.5 (5791) | .64 | |
|
|
|
|
|
|
|
|
Working in a health center | 159 (29.0) | 1547 (30.02) | 103.0 (3) | N/A | <.001 |
|
Self-employed GPsb working alone | 272 (49.5) | 2013 (38.39) | 103.0 (3) | N/A | <.001 |
|
Self-employed GPs working in a group practice | 37 (7) | 1226 (23.38) | 103.0 (3) | N/A | <.001 |
|
Other | 81 (15) | 431 (8.2) | 103.0 (3) | N/A | <.001 |
|
|
|
|
|
|
|
|
Solo (1) | 320 (58.3) | 2155 (41.09) | 67.4 (3) | N/A | <.001 |
|
Small (2-4) | 58 (11) | 920 (17.5) | 67.4 (3) | N/A | <.001 |
|
Medium (5-9) | 60 (11) | 995 (19.0) | 67.4 (3) | N/A | <.001 |
|
Large (10 or more) | 111 (20.2) | 1174 (22.39) | 67.4 (3) | N/A | <.001 |
|
|
|
|
|
|
|
|
Large city | 204 (37.2) | 1944 (37.07) | 1.0 (2) | N/A | .62 |
|
Medium- to small-sized city | 156 (28.4) | 1402 (26.74) | 1.0 (2) | N/A | .62 |
|
Rural town | 189 (34.4) | 1898 (36.19) | 1.0 (2) | N/A | .62 |
aN/A: not applicable.
bGPs: general practitioners.
As recommended by Aldenderfer and Blashfield [
All clustering variables were dichotomous (1: available and 0: not available). In total, 44 dichotomous variables were selected based on their theoretical relationship; that is, they relate to the functional characteristics of 1 of the 4 CISs (EHR, HIE, PHR, or telehealth). We also used 5 measures of performance. Four measures (quality of care provided to patients, efficiency of the practice, productivity of the practice, and improvement of working processes) were based on a 4-point Likert-type scale ranging from 0 (“strongly disagree”) to 3 (“strongly agree”), and 1 measure (number of patients over the last 2 years) of a categorical type with 3 categories (1: decrease, 2: remain the same, 3: increase)
A standardized average frequency or scaled frequency was computed for each group of variables, including CIS capabilities and performance, resulting in 8 scales. As presented in
Reliability of the scales.
Factors | Number of items | Cronbach alpha | |||
|
|||||
|
Electronic health record | 19 | .87 | ||
|
Health information exchange | 15 | .87 | ||
|
Telehealth | 4 | .59 | ||
|
Personal health record | 6 | .80 | ||
|
|||||
|
Quality of care provided to patients | 6 | .92 | ||
|
Efficiency of the practice | 5 | .89 | ||
|
Productivity of the practice | 5 | .83 | ||
|
Improvement of personal working practice | 4 | .80 |
The statistical analysis was performed in 3 parts: (1) cluster analysis, (2) ANOVAs With Tamhane T2 Post Hoc Tests, and (3) regression analysis.
As indicated earlier, we adopted a configurational approach that is taxonomic, based on cluster analysis [
Broadly speaking, cluster analysis is a multivariate statistical technique that classifies items or objects (individuals, firms, or behaviors) from a given population into subgroups or clusters so that items that are classified in the same cluster are more similar to one another than they are to items in another cluster [
It is important to remember that contrary to other statistical techniques such as regression analysis, which necessitate the satisfaction of the linearity assumption and have established rules for sample size calculation [
The optimum number of clusters was determined by inspecting the dendrogram generated in the combination of the Ward minimum variance clustering algorithm and the squared Euclidean distance. This examination revealed that a 3-cluster solution would be optimal. To ascertain the reliability of the solution [
It is worth recalling that cluster analysis was performed with and without the imputation of missing values. Of note is the fact that the results were similar with and without the imputation of missing values. As cluster analyses necessitate all variables to have nonmissing values, 740 observations were not classified in the analysis without the imputation of missing values. The two 3-cluster solutions were then compared to determine the degree of agreement among members of each cluster using Cohen kappa coefficient. The results reveal an almost perfect agreement between the two 3-cluster solutions with a kappa of 0.99 (kappa in the range of 0.80-1.00) [
A multiple discriminant function analysis was performed following a cross-validation approach [
Although our empirically derived taxonomy appears to be meaningful, its quality is discussed in light of Rich [
The selection of clustering variables from which GPs were grouped was theory driven [
The resulting taxonomy is built on the broad foundations of the DOI theory [
The anchoring of the development of our taxonomy in the theory of DOI [
GPs were assigned to specific clusters or groups resulting from an inductive process based on empirical, multivariate data analysis as well as the application of ANOVA and post hoc analysis to enhance the validity of the results.
By deriving the taxonomy from the actual capabilities of the CIS circumscribed by each artifact (EHR, HIE, PHR, and telehealth technologies), we can claim that our taxonomy reflects the real world for both practitioners as well as theorists and depicts the actual landscape of EHR, HIE, PHR, and Telehealth adoption by GPs within the EU.
In a subsequent step, the 3 profiles (clusters) were compared according to 4 performance outcomes (quality of care provided to patients, efficiency of the practice, productivity of the practice, improvement of working processes) using an ANOVA and a Tamhane T2 post hoc test.
We used ANOVAs with Tamhane T2 post hoc tests to compare the 4 performance outcomes that were rated with a Likert scale: quality of care provided to patients, efficiency of the practice, productivity of the practice, and improvement of personal working practice.
This analysis was performed in 4 steps. First, we used a regression model to identify the predictors of CIS adoption.
Second, for each of the following 6 characteristic variables, that is, 4 physician characteristics (gender, age, professional status, and years spent in general practice) and 2 practice characteristics (workplace location and practice size), a univariate multinomial logistic regression was conducted to analyze the effect of the characteristic variable on cluster membership.
Third, the multivariate multinomial logistic regression model was conducted with the 6 characteristic variables as independent variables and the 3-cluster solution as the outcome variable. This was used to analyze the effect of each characteristic variable on cluster membership, as shown in the first step, but controlling for the other 5 characteristic variables.
Fourth, the multiple multinomial logistic regression was performed to see if the cluster membership predicted the level of performance in terms of “the number of patients over the past two (2) years” when controlling for the 6 characteristic variables.
The logistic regression model indicates that compared with GPs who store patient data electronically, those who do not tend to be older in age, self-employed, working alone, with fewer years spent in general practice (
As shown in
Capabilities profile and analysis of variance of clinical information systems.
Variables | Cluster | ANOVAa | ||||
|
1 | 2 | 3 | |||
Number of participants (n=5244), n (%) | 1956 (37.30) | 2764 (52.71) | 524 (10.0) | N/Ab | N/A | |
|
|
|
||||
|
Electronic health record | 5675.8 (2) | <.001 | |||
|
Health information exchange | 2517.1 (2) | <.001 | |||
|
Telehealth | 764.7 (2) | <.001 | |||
|
Personal health record | 2727.4 (2) | <.001 |
aANOVA: analysis of variance.
bN/A: not applicable.
c,d,e,fWithin rows, different superscripts indicate significant (
The strong profile (n=1956) is the second largest of the 3 clusters and accounts for approximately 37% of the sample. Statistically, this cluster scored
The moderate profile (n=2764) is the largest of the 3 clusters and accounts for approximately 53% of the sample. This cluster scored
The weak profile (n=524) is the smallest of the 3 profiles and accounts for approximately 10% of the sample. This cluster scored
Univariate and multivariate logistic regression analyses were performed (
The multivariate model indicated that female GPs were more likely than their male counterparts to be in the
The multivariate logistic model indicated that the GPs working in a health center were less likely than the self-employed GPs working alone to be in the
Similarly, self-employed GPs working in a group practice were found to be less likely than the self-employed GPs working alone to be in the
The multivariate model indicates no association between physician age and membership in the
The multivariate model indicates that senior GPs are less likely to be in the
The multivariate model also indicates that GPs with more years of practice are more likely to be in the
The multivariate model indicates that GPs within a practice located in a medium- to small-sized city or in a rural town are less likely than those located in a large city to be in the
First, no association was found for membership to the
Second, between medium and solo, GPs working in medium-sized practice groups are less likely to be in the
Finally, between large and solo, GPs working in large practice groups are also less likely to be in the
As stated earlier, we used 5 measures of performance, including 4 based on a 4-point Likert-type scale (quality of care provided to patients, efficiency of the practice, productivity of the practice, improvement of working processes) and 1 variable of categorical type that is composed of 3 levels (1: decrease, 2: remain the same, 3: increase) related to the number of patients treated over the last 2 years.
The results in
Clinical information systems profiles and practice performance.
Variables | Cluster | ANOVAa | |||||
|
1 | 2 | 3 | ||||
Number of participants (n=5244), n (%) | 1956 (37.30) | 2764 (52.71) | 524 (10.0) | N/Ab | N/A | ||
|
|
|
|
|
|
||
|
Quality of care provided to patients | 53.7 (2) | <.001 | ||||
|
Efficiency of the practice | 33.3 (2) | <.001 | ||||
|
Productivity of the practice | 41.0 (2) | <.001 | ||||
|
Working processes improvement | 73.1 (2) | <.001 |
aANOVA: analysis of variance.
b N/A: not applicable.
c,d,e,fWithin rows, different subscripts indicate significant (
A multinomial logistic regression model was used to test the association between the 3 profiles and the evolution of the number of patients treated over the past 2 years (a categorical-type variable with 3 categories, ie, 1: decrease, 2: remain the same, 3: increase) by controlling 6 characteristic variables with
Over the past several years, scholars and policy makers have agreed on the unsustainable nature of the increasing trends of health care spending and investing in health information technologies is seen as a viable option in dealing with this threat. As a result, most countries of the Organisation for Economic Co-operation and Development have begun promoting and investing in CISs and making them one of their top priorities. In this context, 4 CISs have emerged as the most important: EHR, HIE, PHR, and telehealth. Although our knowledge of the 4 CISs has been advanced by several studies that have investigated their adoption and associated performance outcomes, the majority did so by considering the 4 CISs in isolation, which implies that our understanding of this complex phenomenon is still limited.
Using data collected by the EC through a survey of 5793 GPs conducted throughout the EU, this study sought to improve our understanding of the adoption of 4 CISs and the implications for performance outcomes. To the best of our knowledge, this study is one of the first to investigate the empirical configurations of the capabilities of the 4 most important CISs (EHR, HIE, PHR, and telehealth) in general practice settings. At the same time, we believe that the findings of this study provide several interesting insights for medical informatics research by confirming, extending, or challenging previous results, in addition to having important normative implications.
First, consistent with previous studies on the adoption of EHR [
Second, after measuring the capabilities associated with each of the 4 CISs, we empirically uncovered 3 theoretically meaningful and significantly well-separated configurations of profiles of CIS adoption by GPs. This result empirically confirms that CIS capabilities as organizational elements correlate in an understandable and stable way [
Third, among the 3 profiles, one, the
Fourth, 2 counterintuitive pictures emerged from our results. First, when scrutinizing the
This research contributes to the configuration literature by responding to a call for further empirical research on equifinality [
From a practical viewpoint, we contend that the resulting profiles of European GPs will assist policy makers to make sense of the general practice adoption of the 4 major CISs. As indicated in our study, GPs have been separated into “discrete and relatively homogeneous groups” [
By investigating the configuration of the 4 most important CISs and the associated implications for performance outcomes, this study explores a topic that has received limited attention until now. Hence, in interpreting our results, one should keep in mind some limitations. First, generalizability may be limited because our sample is composed of only European GPs. Second, there are intrinsic limitations due to the use of secondary data. In fact, we used a data set that was not collected to meet the specific objectives of this study. Third, even though the results of the testing instrument reported in this study indicate high reliability for most scales, one out of the 4 scales measuring CIS capability (telehealth) has a reliability less than 0.6. In addition, even though the instrument has substantial face validity, it has not been subjected to formal psychometric assessment.
Given the paucity of studies that investigate the empirical configurations of the 4 CISs (EHR, HIE, PHR, and Telehealth) at either primary care practice or hospital settings, we encourage other researchers to build upon our results and investigate the configuration of these 4 CISs and the associated implications for performance outcomes in other regions, including hospital settings.
General practitioners' characteristics associated with the Adoption of electronic storage of patient data.
Results of the logistic regression of general practioners and practice characteristics by cluster.
Multinomial logistic regression model to test association between profiles and evolution of the number of patients over the past 2 years, by controlling 6 characteristic variables.
analysis of variance
clinical information system
diffusion of innovation
European commission
electronic health
electronic health record
European Union
general practitioner
health information exchange
information and communication technology
odds ratio
personal health record
The first author contributed to the conception and design of the study and drafted the first version of the manuscript. The third author performed the statistical calculations using SAS software. All authors reviewed and approved this study.
None declared.