Patient Recruitment Into a Multicenter Clinical Cohort Linking Electronic Health Records From 5 Health Systems: Cross-sectional Analysis

Introduction

Background

Recruitment of patients for research using electronic health records (EHRs) has potential to enhance the applicability and efficiency of patient-centered research [1,2]. Despite logistical and ethical challenges with recruiting for research using clinical data sources (eg, EHRs) or clinical communication methods (eg, EHR patient portals) [3], researchers are developing strategies that are responsive to both health system and patient stakeholders [4]. In addition, there is interest in designing research studies that include “real-world evidence” (ie, health care information representative of patients, populations, and health care delivery systems from actual clinical settings [5,6]).

In 2014, the National Patient-Centered Research Network (PCORnet) was launched with funding from the Patient-Centered Outcomes Research Institute (PCORI). PCORnet's infrastructure can support EHR-based recruitment and study implementation [7]. The PaTH Clinical Research Network (CRN) originally brought together 4 academic medical centers in the mid-Atlantic region of the United States to build the infrastructure to share EHR data across health systems so that patient-centered clinical questions could be answered in real-world settings [8]. A fifth academic health center (Geisinger Health System) was added to the network in 2015.

Using EHR data from these 5 health systems, standardized to PCORnet specifications [9], we developed a cohort of primary care and bariatric surgery patients called the “Healthy Lifestyles, Body Weight, and Health Care” cohort. The overall aim of the cohort was to complete online surveys about quality of life, healthy lifestyles, and weight management, which could be linked to selected EHR data. Another explicit goal of the CRN was to utilize several different recruitment methods in order to compare sample demographics and survey response rates by recruitment strategy. In this paper, we describe the enrollment process and then compare 4 different recruitment strategies: (1) in-person recruitment by study staff from clinical sites, (2) US postal mail recruitment letters, (3) secure email, and (4) direct EHR recruitment through secure patient web portals. The objective of this paper was to describe the steps and efficiency of the recruitment process and participant characteristics by strategy.

Methods

Institutional review board approval was obtained from PaTH's single institutional review board at The Johns Hopkins University School of Medicine, Baltimore, Maryland. We benefitted from the input of patient stakeholders in the design of the recruitment methods and online survey. Participants who enrolled in this study were not offered any compensation.

EHR-Based Participant Eligibility Criteria

We recruited adult patients from primary care and bariatric surgery clinics to complete surveys and form an EHR cohort from 5 health care systems included in the PaTH CRN—Geisinger Health System, Johns Hopkins Health System, Penn State Milton S. Hershey Medical Center, Temple Health System, and the University of Pittsburgh Medical Center. We identified participants using EHR-based eligibility criteria (ie, the “computable phenotype”). Eligible patients were aged ≥18 years and had a minimum of 2 weight measurements and 1 height measurement recorded between January 1, 2011, and May 31, 2015. Patients in all BMI categories were eligible. Participants were excluded if they were deceased or non–English proficient, as assessed at the time of consent, because the consent form and survey were only available in the English language.

Participant Recruitment and Setting

Each health system used one or more of the following 4 EHR-based research invitation strategies: (1) in-person recruitment by study staff from partnering outpatient primary care and bariatric surgery clinics; (2) US postal mail recruitment invitation letters using the mailing address listed in the EHR, with an online registration link contained in the invitation; (3) email to potentially eligible participants using the email address stored in the EHR, with an embedded registration link in the email; and (4) secure EHR recruitment messages through the patient web portal with an embedded registration link.

Each health system tailored its own strategy to recruit participants from the large pool of potentially eligible patients identified using the computable phenotype. At each site, local norms for integrating research with clinical practice were followed, leading to differences in the number of patients approached by each strategy per site. Importantly, each site’s study team partnered with specific clinical practices so that recruitment letters were jointly sent from clinical representatives involved in the patients’ care and members of the research team and focused on patients from certain practices. Notably, 2 health systems used direct EHR recruitment through secure patient web portals (sites C and D), while 1 site (site B) was unable to use email because email address was not a data field within the EHR. The content of the recruitment letters and in-person scripts had similar language across all health systems and recruitment strategies.

Each health system had a slightly different recruitment window, but the total recruitment window for all sites was from April 2015 to November 2016. We stopped the recruitment when we met or exceeded the a priori goal of recruiting 1000 participants.

Consent and Enrollment Process

Regardless of the recruitment strategy or method used to complete the baseline survey, all participants completed a web-based consent form, including an initial “consent quiz” designed to ensure that patients understood that they were agreeing to participate in a research study. Participants consented to complete online surveys and for the research study to access their EHR. Enrollment was determined by consenting and having any data entered into the baseline survey. Participants who were recruited in person used an electronic tablet to access the consent form and complete the baseline survey. For participants recruited through the EHR patient portal, the survey was embedded in the EHR, which required that it had to be completed and “submitted” in order to be accessible and counted as enrolled. We created an enrollment flag in the EHR to identify participants who were enrolled in the cohort study.

Data Collection

Participants completed a baseline survey about sociodemographic background, weight management practices, weight-related interactions with the health system, diet, physical activity, and quality of life using standard survey measures [10-12].

We extracted EHR data (eg, laboratory values, blood pressure, anthropometric measurements, diagnosis, and procedure codes) from each site’s PCORNet Common Data Model for all enrolled participants [7,13]. The EHR and survey data were linked for analysis.

Statistical Methods

For this paper, we were primarily interested in describing the patterns of enrollment by recruitment modality, study site, and participant characteristics.

Anthropomorphic data were cleaned to remove unlikely values (eg, BMI ≤15 kg/m² or ≥90 kg/m²; height ≤4 ft or ≥7 ft; weight ≤50 lbs or ≥700 lbs) prior to analyses. We identified the BMI in the EHR data closest to the enrollment date, within a maximum of 3 years, and used a 5-year window to determine comorbidity diagnoses based on International Statistical Classification of Diseases and Related Health Problems codes [14].

We used descriptive statistics (means and standard deviations for continuous measures, frequencies and proportions for nominal measures) to examine differences between the large pool of participants deemed eligible using the computable phenotype and those who enrolled in the study. We defined the “efficiency” of each recruitment strategy as the proportion of the total numbers of participants who enrolled divided by the total number of participants who were approached by the strategy (eg, total enrolled via email divided by total who were sent the email). We performed ANOVA F tests and chi-square tests (or Fisher exact tests, where appropriate) to determine if participant characteristics varied by recruitment modality. Because the 5 health systems preferentially designed and utilized different recruitment modalities, we were not able to assess predictors of enrollment by modality using regression models because of complete collinearity by study site. Two-sided P≤.05 was considered statistically significant. Analyses were performed using SAS statistical software (version 9.4; SAS Institute).

Results

Figure 1 shows the enrollment steps and efficiency of enrollment into the cohort by recruitment modality. We identified 5,051,187 eligible patients using the computable phenotype applied to the EHRs from 5 health systems. Based on each health systems’ own method of partnering with clinics and recruiting participants, a total of 40,048 patients (33,839 patients from primary care clinics and 6209 patients from bariatric surgery clinics) were then sent recruitment messages or approached in person. Across all sites, the patient recruitment strategies were deployed as follows: 442 (1.1%) patients in person; 12,710 (31.7%) patients by postal mail; 25,224 (63.0%) patients by email; and 1672 (4.2%) patients by EHR patient portals. A total of 1185 participants were enrolled in the cohort, with 907 (76.5%) from primary care practices and 278 (23.5%) from bariatric surgery practices. The efficiency of enrollment by recruitment strategy was by far the highest for in-person recruitment (148/442, 33.5%), followed by email (730/25,224, 2.9%) and EHR patient portal (49/1672, 2.9%), with postal mail being least efficient (258/12,710, 2.0%). Overall, 65.7% (779/1185) were enrolled through electronic messaging (email or EHR portal).

‎View this figure

Table 1 compares the demographics and medical conditions between the 1185 participants enrolled in the cohort from primary care and bariatric surgery clinics and the 5,051,187 total patients deemed potentially eligible based on the application of the computable phenotype to their EHRs. Compared with all eligible participants, enrollees were older (aged 58.1 years in primary care clinics and 56.5 years in bariatric surgery clinics versus 43.6 years among all eligible). The bariatric surgery group had the highest proportion of women (207/278, 74.5%) compared with primary care (595/907, 65.6%) and all eligible patients (2,778,178/5,051,187, 55.0%). The proportion of Black individuals among the enrolled patient groups was smaller than that among all patients deemed eligible (81/907, 8.9% enrolled in primary care and 27/278, 9.7% in bariatric surgery versus 558,125/5,051,187, 11.1% from all patients deemed eligible). Overall, very few Hispanic participants were enrolled (14/907, 1.5% in primary care and 1/278, 0.4% in bariatric surgery) compared with the number of Hispanic individuals among the health systems’ eligible participants (127,045/5,051,187, 2.7%). Information regarding level of education was not available for nonenrolled patients; in those enrolled from either primary care or bariatric surgery clinics, more than 85% of participants had at least some college education. The mean BMI was highest for the patients from bariatric surgery clinics (34.3 kg/m²), followed by the patients from primary care clinics (30.2 kg/m²) and the total population of eligible patients (28.3 kg/m²). A high proportion of enrollees had hypertension (507/1185, 42.8%) and diabetes (232/1185, 19.6%), but these conditions were even more common among the large sample of eligible patients who were not enrolled (689,925/5,051,187, 52.1% with hypertension and 407,036/5,051,187, 30.7% with diabetes).

Figure 2 demonstrates the distribution of participants enrolled by strategy and recruitment site from the 5 health systems. Because of differences in research policies, staffing, and capabilities, each health system differed in the number of recruitment strategies used in the primary care and bariatric surgery settings. Table 2 shows the characteristics of enrolled cohort participants by recruitment strategy. The proportion of Black patients who were recruited using EHR patient portals (8/49, 16.3%) and email (80/730, 11.0%) was higher than that with in-person (7/148, 4.7%) and postal (13/258, 5.0%) strategies. We found no statistically significant differences in participants’ level of education or comorbid health conditions between the 4 different recruitment strategies.

‎View this figure

Discussion

This study reports the experience of the PaTH CRN’s recruitment of patients from primary care and bariatric surgery clinics from 5 health systems into a study cohort. Even though the in-person recruitment had the greatest efficiency (33.5%), this strategy required research staff to identify when patients had upcoming appointments and to be available on site. Ultimately, only 442 patients were approached in person and 148 enrolled. The majority of participants were recruited using email (730/1185, 61.6%). Although email recruitment required less staff research time, this strategy was limited by having patients’ email addresses recorded and available in the EHRs. The secure patient portal was a strategy employed by 2 health systems and had a recruitment efficiency comparable to email (2.9% of those who received a message enrolled in the study). The electronic recruitment strategies may be particularly desirable during times when research staff cannot be present in the clinic, such as under COVID-19–related restrictions.

Traditionally, researchers have relied on postal mailings, newspaper or radio advertising, or random-digit dialing methods for survey-based research as well as for recruitment into clinical trials [15]. EHRs provide an avenue for screening and then directly targeting recruitment at potentially eligible patients for research studies [4,15]. Recruitment of patients from EHRs has the potential to be more inclusive by including patients who might be sicker and those who are undergoing care in real-world health care systems [5,16]. However, no method of recruitment is without the potential for selection biases. With EHR-based recruitment, there are concerns about external validity or generalizability to nonpatient or health care populations, particularly for prevention-oriented studies [17]. In addition, patients with reduced access to care or those who are not regularly followed in the health care setting may be less likely to be approached in person (ie, at the time of a clinic visit) or to receive recruitment messages when they do not have an email address or access the patient portal. While 90% of US adults now use the internet, older adults and Black citizens still show lower rates of internet access [18], but technology usage rates are increasing [19]. A recent study by Walker and colleagues [18] showed that fewer Black patients and patients over the age of 70 had access to an inpatient portal. Notably, in our study, which used a combination of recruitment strategies, approximately 9% of enrolled participants identified as Black compared with 11% among patients identified as potentially eligible. In fact, researchers could leverage the EHR to specifically target patient populations by demographics or medical diagnosis (eg, by race, ethnicity, age, or rare health conditions [20,21]).

Despite the increasing need for effective and less staff-intensive methods of recruitment for clinical trials and surveys, few studies have compared recruitment strategies [22,23]. One study reported and compared response rates from various recruitment methods (postal survey, postal invitation to complete an internet survey, and postal invitation for a telephone survey) for an environmental survey but did not include EHR-based recruitment methods [23]. The study showed the highest response rate (30%) for the telephone survey [23]. The EHR provides a new way to identify potential participants for research studies by applying electronic eligibility criteria to sometimes very large pools of potentially eligible patients [24].

A systematic review by Lai and Afseth [25] assessed the effectiveness and efficiency of EHR-based recruitment methods. They identified 13 articles, of which 11 reported recruitment efficiency and most used alerts sent directly to physicians or staff to notify of participant eligibility [25]. EHRs have multiple functionalities to support recruitment [26]. In our study, we utilized the EHR for several purposes: to generate lists of potentially eligible patients using a computable phenotype, to obtain postal mailing addresses and email addresses, and to send recruitment invitations using the secure patient portal at 2 sites. Some academic research centers have designed patient portal recruitment services to enable, but also limit, recruitment using the portal for certain approved studies [1,21,27,28]. In a 2019 single-institution study that included 13 separate EHR-based recruitment strategies using the patient portal recruitment service, the average response rate for patient portal messages was 2.9%, which was the same as our enrollment rate for both email and patient portal recruitment [21]. Interestingly, we offered no compensation to patients to enroll in our study, yet the studies reviewed by Miller et al [21] did offer compensation.

Although the computable phenotype enabled our teams to identify a very large number of patients who were potentially eligible, each health system designed its own outreach methods to patients by targeting specific clinical sites, thereby greatly reducing the potential number of patients that could have been contacted about this research opportunity. Importantly, institutional review boards at each site prohibited the “cold calling” of patients (ie, directly contacting potential research participants based on prior knowledge of the patients’ health information in the absence of a treatment or clinical relationship [29]). Therefore, each site partnered with primary care and bariatric surgery providers, who approached the patients about the study first, either with their signature on the recruitment invitation or through an in-person introduction. However, a recent landscape analysis by McHugh and colleagues [29] highlighted that this universally applied “ban on cold calling” could impose a gatekeeping function and potentially reduce patient autonomy, decreasing access to research and risking the introduction of selection bias into research studies. They suggested alternative approaches to ensuring patient privacy [29], with the goal of broadening access to health research participation [15].

We identified several limitations of this study. First, this was a 5-center, multisite study and the deployed recruitment strategies were dependent on the norms for research recruitment at each site, with the implication that not all samples were directly comparable by site or by strategy. For example, 3 sites did not allow research recruitment using the EHR patient portal and 1 site did not have access to email addresses in the EHRs. Because of the differences in clinical populations’ demographics between sites, there was high correlation between the site’s choice of recruitment methods and the patients who were recruited, limiting our ability to draw conclusions about whether specific recruitment modalities are more (or less) effective for specific racial/ethnic groups or different age categories. Second, the overall response rate was low; however, it is very comparable to response rates from other studies that relied on EHR-based or email-based recruitment. We were able to assess for selection biases by comparing those patients who were deemed eligible with those who enrolled in the study. We showed lower uptake among Latinx patients, but this was in part because the consent and survey were limited to participants able to read in English. Third, although this study was low burden for participants, as they consented to having the research team review medical records and complete a 20-minute online survey, we did not offer any compensation for their time, which could have limited enrollment. Therefore, we are not able to draw conclusions about how effective our recruitment strategies would be for more intensive studies, for studies not using online data collection, or for studies offering participant incentives. Fourth, we were unable to estimate cost or cost-effectiveness of the recruitment strategies or describe in detail the staffing time or corresponding resources involved for each health system, such as the effort for the health informatics team to support EHR-based research or the time for research staff to conduct in-person clinic-based recruitment. Fifth, recruitment for this study occurred approximately 4 years ago (2015-2016) and it is possible that response rates and methods for electronic recruitment have improved over time. Sixth, the 5 health systems were all based in the mid-Atlantic region of the United States, which could limit the generalizability of these findings to other regions of the country and outside the United States.

The major implication of our study is to inform the selection of recruitment strategies for the design of future cohort studies, utilizing the capabilities of the modern-day EHR system. We anticipate that other researchers could find this information useful in the design of their recruitment strategies and to estimate the expected yields from “low touch” strategies that require less personnel contact with potential participants. As health systems and institutional review boards become more comfortable and familiar with EHR-based recruitment, it will be possible to achieve greater consistency between the recruitment processes and even sample selection to reduce biases across systems. Ultimately, to facilitate EHR-based research across multiple settings (eg, inpatient and ambulatory), large health systems will need to invest in infrastructure to support and link smaller clinical centers and subsidize their use of a common EHR. An example of a successful academic and community-based network is the OCHIN network, which provides a research infrastructure using EHR data from a national network of smaller community health centers [30].

In conclusion, we recruited a total of 1085 patients from primary care and bariatric surgery clinics to complete a survey and participate in an EHR-based cohort study using 4 recruitment methods. The greatest recruitment yield was achieved using the email-based method, but the greatest efficiency resulted from in-person recruitment. Implementation of low-resource recruitment approaches has important implications for future patient-centered studies in health system settings.