Patients visiting the UMC Utrecht for the first time for the evaluation of cardiovascular disease or risk factors were eligible for inclusion in the UCC-CVRM LHS. The full rationale of UCC-CVRM has been described elsewhere [9]. The eIC pilot was conducted between November 2021 and August 2022. During this period, all patients (18 years and older) visiting the cardiology outpatient clinic for first-time evaluation automatically received an email. This email notified these patients about UCC-CVRM and the associated eIC form that was available for completion in the UMC Utrecht patient portal. The IC procedures of the face-to-face IC and eIC are illustrated in Figure 1. The full details eIC form presented to the patients in the patient portal are included in Multimedia Appendix 1.

In both the face-to-face IC and the eIC forms, IC was asked for linkage with national registries, GPs, and other hospitals through the following two statements: (1) I consent to future requests to link with various international or national registries, such as the Central Bureau of Statistics (also known as Statistics Netherlands) where all causes of death are registered, the Dutch Cancer Registration (NKR), where all people with cancer are registered, the National Basic Registration of Hospital Care (LBZ), where all hospital admissions are registered, the Foundation for Pharmaceutical Statistics (SFK), where all people who use medicines are registered, and other regional and national registries. (2) I consent to the retrieval of my medical information from my GP, my pharmacy, and any other hospitals where I have been treated in the past.

We collected data from all patients aged 18 years or older referred to a cardiology outpatient clinic. We used the Utrecht Patient Oriented Database to collect data from the patients who participated in the eIC pilot, referred to as the “eIC cohort.” The Utrecht Patient Oriented Database comprises data on, among others, patient characteristics and laboratory tests for all patients treated at the UMC Utrecht since 2004 [14]. We collected routine care data related to the patient’s demographics and cardiovascular risk, namely blood pressure, BMI, and laboratory measurements (serum lipids, glycated hemoglobin [HbA_1c], hemoglobin, and renal function). These data were also collected for the cardiology patients who were invited during the period in which a face-to-face IC procedure was in place, referred to as the “face-to-face IC cohort.” From the face-to-face IC cohort, only patients invited up until December 31, 2019, were included because the COVID-19 pandemic significantly hampered the face-to-face IC procedure.

All measurements were extracted from structured fields in the EHR. Blood pressure values were extracted from the EHR ±7 days from the date of the visit at the cardiology outpatient clinic. For other measurements, the closest value, within ±21 days of the visit date, was extracted. If no measurements were found within these cutoffs, the measurement was considered missing. An overview of missingness per variable is added as Multimedia Appendix 2. Age was calculated by subtracting the date of the visit from the patient’s date of birth. The estimated glomerular filtration rate was calculated using the Chronic Kidney Disease Epidemiology Collaboration equation and used as a measure of renal function [15].

We presented the yield for both the eIC and face-to-face IC cohort as counts and the percentages of patients who, (1) consented to the linkage of their data with their GP, pharmacy, and other hospitals, and linkage with national registries (ie, “full consent”); (2) did not consent for the linkage of their data with their GP, pharmacy, and other hospitals, nor to the linkage with national registries (ie, “nonconsent”); (3) consented to only 1 of the 2; and (4) the percentage of patients who did not respond at all or provided an answer for only 1 of the 2 statements (ie, “nonresponse”). Due to the limited number of observations in some IC response groups (n<25), especially in the eIC cohort, other than the full consent (n=1291) and nonresponse (n=1477) group, further analyses were restricted to the full consent and nonresponse group only.

To assess differences in patient groups, characteristics of the patients with full consent were stratified by cohort (ie, eIC cohort vs face-to-face IC cohort). As a supplement, we also explored the characteristics of the nonresponders by cohort. Finally, we assessed the differences in patient characteristics between the response categories within each cohort (ie, full consent versus nonresponse).

Clinical characteristics were presented as means with corresponding SDs, medians with corresponding interquartile ranges, or counts and percentages, as appropriate. To quantify differences in characteristics between cohorts, or response categories within cohorts, we performed multivariable linear regression analyses, adjusted for age, categorized into 4 categories with an approximately equal number of observations (18 to 47, 48 to 60, 61 to 70, and 71 to 95 years old), and sex. The assumptions of linear regression (eg, approximate normal distribution of the error terms, homoscedasticity of errors) were assessed. Where needed, we used the Box-Cox method to estimate the most appropriate transformation of the dependent variable to stabilize the variance and improve the accuracy of our estimations [16]. Similarly, multivariable linear regression was used to assess the difference in age (as a continuous variable) between groups, adjusted for sex. Multivariable logistic regression was used to assess the difference in sex between the groups (ie, between the 2 cohorts and between the response categories within each cohort), adjusted for age (categorized into 4 categories with an approximately equal number of observations). As a sensitivity analysis, we repeated the adjusted regression analyses with age as a continuous variable, to assess whether the categorization led to different results.

We used the Bonferroni correction to reduce the risk of a type I error resulting from the multiple tests [17]. Thus, the α that we considered as cutoff, .05, was divided by the number of analyses (N=13) performed per comparison. Therefore, a P value ≤.0038 was considered statistically significant.

All statistical analyses were performed using R software (version 4.0.5; The R Foundation) [18].

We obtained an additional ethical waiver (number 19/641) from the Research Ethics Committee Utrecht to examine the characteristics of patients in all IC response categories. Patients who objected to the use of their clinical data for research purposes via the UMC Utrecht opt-out procedure were excluded from this study. Data were pseudonymized and the patients did not receive any compensation for their participation in this study.

In total, 3139 patients participated in this study, of whom 885 (28.2%) participated in the eIC pilot cohort and 2254 (71.8%) in the face-to-face IC cohort (Multimedia Appendix 3). Of all patients from the eIC cohort, 49.9% (442/885) completed the eIC form, 50.1% (443/885) did not respond. Of all patients who completed the eIC form, we obtained full consent for linkage with GPs, hospitals, and national registries from 93.9% (415/442) of the patients. In the face-to-face IC cohort, 54.1% (1220/2254) of all patients completed the IC form, and 45.9% (1034/2254) patients did not respond. The percentage of responding patients with full consent was higher in the eIC cohort as compared with the face-to-face IC cohort (415/442, 93.9% vs 876/1220, 71.8%, respectively).

Overall, fully consenting patients had similar patient characteristics (Table 1). Adjusted for sex and age, the eIC cohort had lower hemoglobin levels and higher HbA_1c levels than the face-to-face IC cohort, reaching the multiple testing threshold for statistical significance for hemoglobin (P=.0021).

Similarly, we compared the (clinical) characteristics of the nonresponding patients between cohorts, shown in Multimedia Appendix 4. Adjusted for sex, the nonresponders of the eIC cohort were significantly younger and had, adjusted for age and sex, lower c-reactive protein values than the nonresponders of the face-to-face cohort. No other differences were observed.

Within each cohort, we assessed whether there were differences in characteristics between the response categories (ie, full consent vs nonresponse). In the eIC cohort, the nonresponse group was significantly younger than the full consent group (Table 2). Other than that, the clinical characteristics of the full consent group were similar to those of the nonresponse group.

More differences were found between the response categories of the face-to-face IC cohort. Adjusted for age and sex, patients in the full consent group had higher hemoglobin, but lower HbA_1c and c-reactive protein values than the nonresponse group (Table 3).

We repeated the regression analyses adjusted for sex and age. In these regression analyses, age was maintained continuous instead of categorized, to assess whether the categorization of age led to different results. The results were similar (Multimedia Appendix 5).

We showed that by using an eIC in an LHS, patients more often provided full consent to link their data to national registries, GPs, and other hospitals compared with a face-to-face IC procedure. The clinical characteristics of patients with full consent remained largely similar after changing the IC procedure to an eIC. Except for age, we did not find any differences between the response categories of the eIC cohort, whereas in the face-to-face cohort, several differences were found. These differences potentially suggest a higher (cardiovascular) disease burden in the nonresponse group compared with the full consent group, indicative of a potentially more pronounced selection in the face-to-face approach.

A possible explanation for the differences in characteristics between the response categories in the face-to-face cohort is that patients may have been too ill or frail to attend the physical appointment with the research nurse to discuss and sign the IC form, resulting in nonresponse. The inability to attend the appointment was probably less of an issue in the eIC cohort, as patients were able to access the eIC form remotely. The finding suggests that the use of eIC results in a study population (ie, those who give full consent) that is more representative of the full target population. Our findings agree with a previous study showing that providing computer-based clinical study information leads to more willingness to participate [19], as the increased willingness to participate is consistent with the higher full consent rates found in the eIC group compared with the face-to-face IC group in our study.

Concerns have been raised about whether consent given via an eIC is truly an “informed” consent [8]. According to the principles of the Declaration of Helsinki [7], potential participants must be adequately informed about various aspects of the study, such as its purpose, sources of funding, the anticipated benefits and potential risks, and the right to refuse or withdraw consent to participate without giving a reason [7]. According to previous research, comprehension assessment is more challenging when an eIC procedure is used as there is no direct interaction between the potential participant and researcher [6]. As a result, patients might provide consent without fully understanding what they are consenting to, or, conversely, patients may be less likely to consent because of the lack of personal interaction with the researcher or clinician, especially those who were already doubtful about participating in the first place. However, our findings indicate that the latter might not have been the case in our pilot study, as we observed a higher percentage of patients with full consent in the eIC cohort compared with the face-to-face IC cohort.

Another frequently mentioned concern is that studies using an eIC procedure could become inaccessible to patients who lack the digital literacy needed to access and understand the eIC form [6]. In 2021, the Netherlands had the highest percentage (ie, 79%) of 17- to 74-year-olds with at least basic digital skills in Europe [20]. Therefore, incomprehension of the eIC due to limited digital literacy may appear less of an issue in our study. However, the percentage of persons with basic digital skills varied considerably by age, with older people being less literate [20]. A sensitivity analysis showed that the age distribution of responding patients was similar between the eIC and the face-to-face approach (Multimedia Appendix 6), indicating that the eIC was not less accessible than the face-to-face IC for certain age groups. However, accessibility may be an issue for geriatric patients, who are generally older than cardiology patients and often have geriatric syndromes that sometimes affect comprehension and literacy [21]. These syndromes generally make it difficult to obtain IC from the elderly [21]. eIC could, therefore, also be seen as an opportunity. Unlike paper-based ICs, multiple formats can be used to inform the patient about the purpose of the eIC and to provide technical support, for example, by using instructional videos or audio. The use of multiple formats in IC forms for the elderly has been recommended by, among others, Barron et al [22]. Furthermore, UCC-CVRM’s eIC form is available in UMC Utrecht’s long-existing patient portal. In the portal, patients have the opportunity to, among others, ask questions to their clinician via an e-consult, which can be used if parts of the eIC are unclear [23]. Another possibility would be a hybrid format, allowing patients who prefer correspondence by regular mail to respond using a paper-based IC form. However, it is questionable whether this would be helpful and it would negate the positive aspects of the eIC highlighted in this study (eg, less pronounced selection).

Since July 2022, eICs have been permitted in the Netherlands when certain conditions are met [24]. A total of 6 conditions are described in the guideline written by the Central Committee on Research Involving Human Subjects (Centrale Commissie Mensgebonden Onderzoek) and the Dutch Association of Medical Research Ethics Committees (Nederlandse Vereniging voor Medisch-Ethische toetsingscommissies) [25]. The most important conditions are (1) eIC must be appropriate for the study, meaning that the study is associated with low potential risk and burden for the patient, (2) the eIC process must be sufficiently reliable and confidential, guaranteed by an electronic system that is compliant to the Dutch General Data Protection Regulation (UAVG in Dutch) and ensures the validity of the electronic signatures, and (3) the eIC procedure must be described in the study protocol [24,25]. The implementation of an eIC seems appropriate in the case of the UCC-CVRM, as no potential risk or burden for the patient is involved. Furthermore, in the eIC of the UCC-CVRM, data security, identity verification, and the validity of the electronic signature are ensured by the Dutch digital ID, an identification method for accessing web-based services [26]. Regarding the third condition, an amendment to the UCC-CVRM approach, including the eIC, was submitted and approved by the Research Ethics Committee.

Based on the results of our study, the use of eIC to obtain IC might be a sustainable and adequate way to enable researchers to link with national registries, GPs, and other hospitals. The use of the eIC seemed to have resulted in a population with consent that is more similar to the target population compared with the face-to-face IC, which is of great importance in an LHS. Results from the LHS would be more generalizable to the target population, namely to all patients at higher cardiovascular risk. Yet, one may argue whether ≈50% response to both the electronic and face-to-face IC for an LHS approach is sufficient. In addition, it should be noted that the extractability of CVRM indicators from structured fields in the EHR was much lower in the eIC cohort compared with the face-to-face IC cohort. Groenhof et al [13] showed that the former, protocolized, face-to-face UCC-CVRM approach led to more systematic registration of the cardiovascular risk profile in the EHR, which had a positive effect on CVRM guideline adherence in consenting patients, compared with the situation before UCC-CVRM was introduced [13]. The substantial missingness in the eIC cohort of our study may suggest that these improvements are at risk when the approach is automated, as deviations from the initial protocol are made, potentially leading to suboptimal CVRM in clinical care.

Exploring the views and experiences of patients could help to further improve the eIC form. Therefore, we recommend further qualitative research into the accessibility and understandability of eICs used for similar purposes and in similar settings as the UCC-CVRM LHS from a patient’s perspective.

To the best of our knowledge, we are among the first to investigate the differences in clinical patient characteristics between response categories of an eIC compared with those of a traditional face-to-face IC, specifically in the context of a cardiovascular LHS in a large sample of patients. Our uniqueness, however, limits the ability to compare our findings to the literature, as most research on eIC has focused on user perspectives, experiences, and the ethical considerations of eICs. For example, Chen et al [5] showed that in most included studies, participants had a better understanding of the information when using an eIC compared with a traditional paper-based face-to-face IC, while others found no difference [5]. Nevertheless, they [5] and others [2,6,27] indicated that face-to-face interaction should remain part of the IC process, especially for more complex and higher-risk studies. However, as the UCC-CVRM LHS is not a complex or high-risk study, the face-to-face interaction may be less necessary. Furthermore, the nonresponders in the eIC cohort may not be fully comparable to the nonresponders in the face-to-face IC cohort because, in the eIC cohort, patients received the eIC after their appointment at the cardiology outpatient clinic, whereas in the face-to-face IC cohort, cardiology patients were identified as eligible and received information about the UCC-CVRM LHS prior to their appointment. This means that patients who, for example, canceled their appointment at the last minute would still be included in the face-to-face cohort as nonresponders. It may be that patients who did not attend their appointment at all had different characteristics to those who attended but did not respond to the eIC, potentially affecting the validity of the comparisons made. Finally, the eIC form was piloted in the patient population of the cardiology outpatient clinic only. Although our results indicated that there were only minor differences (ie, hemoglobin) between patients providing full consent using the eIC compared with the face-to-face IC, it remains to be seen whether this would still be the case after implementation of the eIC in other clinical departments.

To conclude, our findings suggest that using an eIC may lead to a better representation of the target population by consenting patients. This increases the generalizability of results from studies using the data collected within the LHS from consenting patients.