Over the past several years, a consensus has emerged on the recognition of the potential of clinical information systems (CISs) to improve the health care delivered to patients and save lives [1,2].

Electronic health records (EHRs) are at the heart of the reform of health systems in many developed countries [3] as well as middle-income countries such as Brazil [4] or India [5]. An EHR can be defined as “an electronic record of health-related information on an individual that conforms to nationally recognized interoperability standards, and that can be created, managed, and consulted by authorized clinicians and staff across more than one healthcare organization” [6]. EHRs can “help address the dual problems of high costs and poor quality in health” [7]. For instance, EHRs can assist clinicians to improve the care provided to patients by promoting adherence to guidelines [8], improving medical practice management, saving time, and facilitating condition-specific queries, to name a few [9]. In the specific context of primary health care organizations or general practices, EHRs offer “a unique opportunity to collect a wide range of ecologically valid patient data to support understanding of disease burden and health trajectories over the life-course” [10].

Due to their potential benefits, previous decades have witnessed rapid growth in the adoption of EHRs in health care settings. However, despite considerable investments by governments, the adoption of EHRs in some primary care organizations has been slow, especially in small practices [11], and disparities have been observed in terms of benefits associated with CISs in primary care practices [12]. In addition, other researchers have underscored that the potential benefits of EHRs are limited when health information stored in the system is shared only within the host institution, which means that greater benefits will be realized if only health information is shared beyond the host institution [13,14] and the technology used to support such sharing is health information exchange (HIE). For this reason, the Meaningful Use program in the United States includes incentives for health care providers to participate in HIE [15]. Empirical evidence suggests that the exchange and sharing of patient data can decrease mortality, systemic costs, and utilization costs in the emergency department [13,16,17].

In addition to the abovementioned CISs (EHR and HIE), the personal health record (PHR) has recently been gaining attention because of its potential to support the transformation of health systems to a more patient-centered model of care [18] as well as its key role in patient engagement [19,20]. Indeed, patient engagement is recognized as a critical factor for improving the quality of care [19] and patient safety [19,21,22]. It is not surprising, then, that patient engagement measures and the sharing of health information between providers and patients are also key parts of the Meaningful Use program in the United States [18,23]. Although the role of patients in the health care process is being increasingly recognized, Krist and Woolf [24] have pointed out that much of the energy of the health information movement has been devoted to the use of health information by clinicians, even though patients’ use of these technologies carries equal promise. One of the most effective ways to share electronic health data with patients is via PHRs.

Similarly, telehealth has been gaining attention because of its potential to reduce barriers to access health care and to save time and reduce costs for remote patients [25,26]. In addition, telehealth can “be clinically supportive and educative by facilitating contact with peers” and, in turn, education can enhance the quality of care provided to patients [27,28].

In conclusion, EHR, HIE, PHR, and telehealth can be considered the most important components of a modern and desirable CIS for both hospital and primary care practices. However, in previous research, these 4 CISs have been studied in isolation with little or no attention to their combination and associated implications for performance outcomes. A search in Medical Literature Analysis and Retrieval System Online (MEDLINE); March 2020) using the 4 terms, “electronic health records,” “health information exchange,” “personal health record,” and “tele-health,” in all fields yielded no hits. The same search performed in Article title, Abstract, Keywords in Scopus (March 2020) yielded no hits. We conclude that there is some evidence from which one can infer that the body of knowledge in the way capabilities associated with the 4 CISs empirically coexist to form configurations or profiles, and the associated implication for performance outcomes is very limited. Of note is the fact that most studies have investigated the 4 CISs in isolation, including their implications for performance outcomes. For instance, several current reviews have found ambivalent or no significant relationship between EHRs and performance outcomes in primary care settings [29,30]. For their part, Black et al [29] concluded that “there is a large gap between the postulated and empirically demonstrated benefits of eHealth technologies [such as EHR, HIE, PHR, and telehealth].”

This paper takes a configurational perspective. In a broad sense, configuration is defined as “any multidimensional constellation of conceptually distinct characteristics that commonly occur together” [31].

Following Miller [32]; Fiss, Marx, and Cambré [33]; and Delery and Doty [34], we argued that studying the configurations of CIS adoption by primary care practices will not only allow us to take a holistic view of the adoption of CIS by these organizations as it is these patterns or profiles of 4 CISs rather than single isolated CISs that are related to performance, but it will also help to reveal insights that would have been otherwise difficult to obtain. In addition, configuration approaches help to investigate equifinality—defined as a concept in which different primary care practice profiles with different configurations of CIS adoption arrive at the same level of the outcome measure in terms of practice performance outcomes [35].

This study has 4 primary goals. The first is to identify predictors of the adoption of clinical systems by general practitioners (GPs) in Europe. The second is to identify and characterize GPs by uncovering typical profiles or patterns of the combination of 4 major CIS capabilities: EHRs, HIEs, PHRs, and telehealth. Consequently, this second objective is inductive in nature, empirically based, and taxonomic, dedicated to classification and subdivision [36,37]. The third objective is to identify physician and practice characteristics that predict cluster or profile membership. Finally, given that the ultimate objective of investing in CISs is to improve the quality of care provided to patients while decreasing cost, the fourth objective is to assess the variation in levels of performance outcomes associated with practice in each configuration or profile.

We used a data set provided by the European Commission (EC) from the 2018 survey of European GPs. The objective of this study was to understand and measure the actual adoption and use of information and communication technology (ICT) and electronic health (eHealth) applications by general practitioners (GPs) in the 27 countries of the European Union (EU27) as well as changes in uptake over time. The 2018 survey of European GPs was a follow-up study of the 2013 survey, which included the EU27 plus 4 other countries (Croatia, Iceland, Norway, and Turkey).

Given that this 2018 survey of European GPs was a follow-up of the 2013 survey, eHealth is broadly defined, as in the previous study, as “the use of Information and Communication Technologies (ICT) across the whole range of health care functions” [38]. The 2018 survey used the same methodological approach and questionnaire as the previous study. Data collection was based on a survey in the EU27 using mixed methodology (web-based; web CATI, where CATI stands for computer-assisted telephone interviews; and face-to-face) and was conducted between January and June 2018 [38,39]. The data collection process was endorsed by the European Union of General Practitioners. A detailed description of the methodological approach is provided only in the 2013 survey report. The EC team took several measures to ensure the representativeness of the sample. The details are provided in the Technical Compendium of 121 pages published by the EC and available to the public [38]. The identification of the universe of the survey was based on the definition from the World Organization of Family Doctors Europe that characterizes GPs in European countries as “specialist physicians trained in the specialty of primary care who ‘exercise their professional role by promoting health, preventing disease and providing cure, care, palliation and promoting patient empowerment and self-management’.” The universe was composed of 425,622 GPs [39].

The questionnaire was composed of 3 parts: (1) GPs’ sociodemographics, organizational settings, practice location, description of tasks, and workload; (2) ICT availability and use within a GP practice that is divided into 4 categories: EHRs, HIE, telehealth, and PHRs; and (3) attitudinal questions as well as questions related to motivations, perceived barriers, and impacts of ICT.

Following the previous 2013 survey approach, a final sample of 5793 GPs was randomly selected over the analyzed EU27, with an overall sampling error of plus or minus 1.30% [39].

Given that the 4 objectives of this study are related to CIS adoption and associated with the implications for performance outcomes, out of the initial sample of 5793 GPs, only the 5244 who had an EHR system and stored patient data electronically were considered for a subsequent analysis. Due to the presence of missing values in 3 variables (HIE, PHR, and telehealth), we applied a multiple imputation strategy. Among the 5244 subjects, 5022 (95.77%) subjects had complete data, 100 (1.91%) subjects had missing values on telehealth only, 72 (1.37%) subjects had missing values on PHR only, and the remaining 50 (<1%) subjects had other missing patterns.

We imputed the missing values using a multiple imputation procedure. On the one hand, multiple imputation methods perform better than single imputation ones. We selected the multiple imputation method of fully conditional specification with the Proc MI procedure in SAS software version 9.4 (SAS Institute) because of its flexibility in allowing us to define the multivariate model by a series of conditional models, one for each incomplete variable [40]. The 4 variables related to each CIS were chosen for the imputation model: EHR, HIE, telehealth, and PHR. For each of the 4 variables, we then chose regression because the regression model allows the specification of a minimum and a maximum value for imputed values on a variable-by-variable basis [41].

On the other hand, it is impossible to use multiple imputed data sets for cluster analysis as it produces different results of clusters, and it seems that there are no methods to combine these results. Therefore, we used the multiple imputation method as discussed but only chose one as the number of imputations. The final 4 variables used in the cluster analyses were the ones after imputations and then standardization: xc_EHR_imp_std, xc_HIE_imp_std, xc_TeH_imp_std, xc_PHR_imp_std.

Although the final 4 variables used for the cluster analysis were the ones with an imputation, we performed the same cluster analysis with the same variables without the imputation of missing values.

As stated earlier, the 549 GPs who did not store patients’ data electronically and those who did not have an EHR were excluded from the analysis. The final sample in our analysis was 5793 – 549 = 5244 subjects. Table 1 compares the characteristics of the 549 and 5244 subsamples. Concerning the total sample (N=5244) composition in terms of countries, 8 countries represent a little less than 50% (49.77%) of the sample, each country accounting for about 5.54% and 7.15% of the sample, respectively. These countries include France, Italy, Poland, Portugal, Romania, Spain, the United Kingdom, and the Netherlands.

As recommended by Aldenderfer and Blashfield [42] and Hair et al [37], the selection of clustering variables is theory driven. The selection of variables was based on the fact that they have a common theoretical foundation—they relate to the functional characteristics or capabilities of 1 of the 4 CISs (EHR, HIE, PHR, and telehealth). Indeed, according to the diffusion of innovation theory (DOI) [43], the characteristics of innovation, such as the functional capabilities of CISs are important factors in explaining its adoption.

All clustering variables were dichotomous (1: available and 0: not available). In total, 44 dichotomous variables were selected based on their theoretical relationship; that is, they relate to the functional characteristics of 1 of the 4 CISs (EHR, HIE, PHR, or telehealth). We also used 5 measures of performance. Four measures (quality of care provided to patients, efficiency of the practice, productivity of the practice, and improvement of working processes) were based on a 4-point Likert-type scale ranging from 0 (“strongly disagree”) to 3 (“strongly agree”), and 1 measure (number of patients over the last 2 years) of a categorical type with 3 categories (1: decrease, 2: remain the same, 3: increase)

A standardized average frequency or scaled frequency was computed for each group of variables, including CIS capabilities and performance, resulting in 8 scales. As presented in Table 2, in 7 out of 8 scales, the measurement of reliability is greater than 0.7 [44]. Although the reliability of telehealth was less than 0.7, we decided to keep this factor because Cronbach’s alpha is sensitive to the number of items in the scale and generally tends to underestimate the internal consistency reliability” [45] and “researchers can rely on less stringent reliability measurements for scales consisting of a few items” [46]. We then used standardized average frequencies as the clustering variables.

The statistical analysis was performed in 3 parts: (1) cluster analysis, (2) ANOVAs With Tamhane T2 Post Hoc Tests, and (3) regression analysis.

As indicated earlier, we adopted a configurational approach that is taxonomic, based on cluster analysis [31,47]. General practices were classified to reveal patterns based on the available CIS capabilities related to EHR, HIE, PHR, and telehealth. Thereafter, we explored the influence of general practitioner and practice characteristics on the availability of the 4 components of CISs included in this study (EHR, HIE, PHR, and telehealth).

Broadly speaking, cluster analysis is a multivariate statistical technique that classifies items or objects (individuals, firms, or behaviors) from a given population into subgroups or clusters so that items that are classified in the same cluster are more similar to one another than they are to items in another cluster [37,48]. In doing so, the homogeneity of objects within each group is maximized, whereas the heterogeneity between the groups is maximized. Cluster analysis is a completely empirical method of classification that is inductive by nature [49]. Following previous research recommendations [48-50], our procedure was based on 2 steps that allowed the determination of the natural number of clusters in the data set, as it has been found to be more effective than other approaches in the ability to recover clusters. First, a hierarchical algorithm was employed to identify and define natural clusters and associated centroids. These centroids were then used as initial seeds in a nonhierarchical algorithm. Later, a discriminant function analysis was carried out [49] to validate the cluster solution. Several studies have used this technique to validate the results of cluster analysis [51,52].

It is important to remember that contrary to other statistical techniques such as regression analysis, which necessitate the satisfaction of the linearity assumption and have established rules for sample size calculation [53], no such assumptions are necessary for cluster analysis [54]. Given that this is a data mining technique, cluster analysis does not have hard sample size rules and does not need to satisfy parametric or even nonparametric statistical test assumptions [55]. Nonetheless, Hair et al [56] underscored that “the sample size must be large enough to provide sufficient representation of small groups within the population and represent the underlying structure.” For their part, Formann [57,58] as well as Dolnicar [59] contend that there is a consensus that the minimal sample size for cluster analysis is 2^k observations (k=number of variables) to achieve sufficient power and confidence in the statistical analysis. In the same vein, Lowry et al [55], including Dolnicar [59], indicate that from a conservative perspective, the minimal sample must be no less than 5×2^k. In this study, k=4, which requires a minimum sample size of 80 to meet this criterion, which is obviously less than the sample size of this study (N=5244).

We used ANOVAs with Tamhane T2 post hoc tests to compare the 4 performance outcomes that were rated with a Likert scale: quality of care provided to patients, efficiency of the practice, productivity of the practice, and improvement of personal working practice.

This analysis was performed in 4 steps. First, we used a regression model to identify the predictors of CIS adoption.

Second, for each of the following 6 characteristic variables, that is, 4 physician characteristics (gender, age, professional status, and years spent in general practice) and 2 practice characteristics (workplace location and practice size), a univariate multinomial logistic regression was conducted to analyze the effect of the characteristic variable on cluster membership.

Third, the multivariate multinomial logistic regression model was conducted with the 6 characteristic variables as independent variables and the 3-cluster solution as the outcome variable. This was used to analyze the effect of each characteristic variable on cluster membership, as shown in the first step, but controlling for the other 5 characteristic variables.

Fourth, the multiple multinomial logistic regression was performed to see if the cluster membership predicted the level of performance in terms of “the number of patients over the past two (2) years” when controlling for the 6 characteristic variables.

The logistic regression model indicates that compared with GPs who store patient data electronically, those who do not tend to be older in age, self-employed, working alone, with fewer years spent in general practice (Multimedia Appendix 1). Of note is the fact that the odds ratios (ORs) for age and years spent in general practice are close to 1.

As shown in Table 3, the ANOVA F test is highly statistically significant for all 4 factors or groups of functional capabilities. In addition, the Tamhane T2 post hoc multiple pairwise comparison test revealed significant differences between the means of all 4 factors or groups of functional capabilities across the 3 clusters [66]. It is important to recall that, contrary to other tests such as the Tukey Honestly Significant Difference Test, the Tamhane T2 post hoc test does not assume equal variances.

Univariate and multivariate logistic regression analyses were performed (Multimedia Appendix 2) with CIS profiles as the outcome and 6 independent variables of physician and practice characteristics: gender, age, physician professional status, workplace location, years spent in general practice, and practice size). Of note is the fact that the outcome variable has 3 CIS profiles, and the odds in the Multimedia Appendix are to the strong profile as the reference.

As stated earlier, we used 5 measures of performance, including 4 based on a 4-point Likert-type scale (quality of care provided to patients, efficiency of the practice, productivity of the practice, improvement of working processes) and 1 variable of categorical type that is composed of 3 levels (1: decrease, 2: remain the same, 3: increase) related to the number of patients treated over the last 2 years.

The results in Table 4 indicates that the strong profile outperforms the other 2 in terms of the quality of care provided to patients, efficiency of the practice, productivity of the practice, and improvement of working processes. Surprisingly, however, the moderate and weak profiles do not differ from one another in terms of the quality of care provided to patients, efficiency of the practice, and productivity of practice, whereas the moderate profile outperforms the weak profile concerning improvement of working processes.

A multinomial logistic regression model was used to test the association between the 3 profiles and the evolution of the number of patients treated over the past 2 years (a categorical-type variable with 3 categories, ie, 1: decrease, 2: remain the same, 3: increase) by controlling 6 characteristic variables with stable level as the reference for the 3-category outcome variable (Multimedia Appendix 3). This model indicates that physicians in the weak profile are more likely to have experienced a decrease in the number of patients treated than those in both the strong and moderate profiles, with OR 1.67 (95% CI 1.20-2.33) and OR 1.82 (95% CI 1.33-2.48), respectively.