Institutional review board approval was obtained from PaTH's single institutional review board at The Johns Hopkins University School of Medicine, Baltimore, Maryland. We benefitted from the input of patient stakeholders in the design of the recruitment methods and online survey. Participants who enrolled in this study were not offered any compensation.

Figure 1 shows the enrollment steps and efficiency of enrollment into the cohort by recruitment modality. We identified 5,051,187 eligible patients using the computable phenotype applied to the EHRs from 5 health systems. Based on each health systems’ own method of partnering with clinics and recruiting participants, a total of 40,048 patients (33,839 patients from primary care clinics and 6209 patients from bariatric surgery clinics) were then sent recruitment messages or approached in person. Across all sites, the patient recruitment strategies were deployed as follows: 442 (1.1%) patients in person; 12,710 (31.7%) patients by postal mail; 25,224 (63.0%) patients by email; and 1672 (4.2%) patients by EHR patient portals. A total of 1185 participants were enrolled in the cohort, with 907 (76.5%) from primary care practices and 278 (23.5%) from bariatric surgery practices. The efficiency of enrollment by recruitment strategy was by far the highest for in-person recruitment (148/442, 33.5%), followed by email (730/25,224, 2.9%) and EHR patient portal (49/1672, 2.9%), with postal mail being least efficient (258/12,710, 2.0%). Overall, 65.7% (779/1185) were enrolled through electronic messaging (email or EHR portal).

Table 1 compares the demographics and medical conditions between the 1185 participants enrolled in the cohort from primary care and bariatric surgery clinics and the 5,051,187 total patients deemed potentially eligible based on the application of the computable phenotype to their EHRs. Compared with all eligible participants, enrollees were older (aged 58.1 years in primary care clinics and 56.5 years in bariatric surgery clinics versus 43.6 years among all eligible). The bariatric surgery group had the highest proportion of women (207/278, 74.5%) compared with primary care (595/907, 65.6%) and all eligible patients (2,778,178/5,051,187, 55.0%). The proportion of Black individuals among the enrolled patient groups was smaller than that among all patients deemed eligible (81/907, 8.9% enrolled in primary care and 27/278, 9.7% in bariatric surgery versus 558,125/5,051,187, 11.1% from all patients deemed eligible). Overall, very few Hispanic participants were enrolled (14/907, 1.5% in primary care and 1/278, 0.4% in bariatric surgery) compared with the number of Hispanic individuals among the health systems’ eligible participants (127,045/5,051,187, 2.7%). Information regarding level of education was not available for nonenrolled patients; in those enrolled from either primary care or bariatric surgery clinics, more than 85% of participants had at least some college education. The mean BMI was highest for the patients from bariatric surgery clinics (34.3 kg/m²), followed by the patients from primary care clinics (30.2 kg/m²) and the total population of eligible patients (28.3 kg/m²). A high proportion of enrollees had hypertension (507/1185, 42.8%) and diabetes (232/1185, 19.6%), but these conditions were even more common among the large sample of eligible patients who were not enrolled (689,925/5,051,187, 52.1% with hypertension and 407,036/5,051,187, 30.7% with diabetes).

Figure 2 demonstrates the distribution of participants enrolled by strategy and recruitment site from the 5 health systems. Because of differences in research policies, staffing, and capabilities, each health system differed in the number of recruitment strategies used in the primary care and bariatric surgery settings. Table 2 shows the characteristics of enrolled cohort participants by recruitment strategy. The proportion of Black patients who were recruited using EHR patient portals (8/49, 16.3%) and email (80/730, 11.0%) was higher than that with in-person (7/148, 4.7%) and postal (13/258, 5.0%) strategies. We found no statistically significant differences in participants’ level of education or comorbid health conditions between the 4 different recruitment strategies.

This study reports the experience of the PaTH CRN’s recruitment of patients from primary care and bariatric surgery clinics from 5 health systems into a study cohort. Even though the in-person recruitment had the greatest efficiency (33.5%), this strategy required research staff to identify when patients had upcoming appointments and to be available on site. Ultimately, only 442 patients were approached in person and 148 enrolled. The majority of participants were recruited using email (730/1185, 61.6%). Although email recruitment required less staff research time, this strategy was limited by having patients’ email addresses recorded and available in the EHRs. The secure patient portal was a strategy employed by 2 health systems and had a recruitment efficiency comparable to email (2.9% of those who received a message enrolled in the study). The electronic recruitment strategies may be particularly desirable during times when research staff cannot be present in the clinic, such as under COVID-19–related restrictions.

Traditionally, researchers have relied on postal mailings, newspaper or radio advertising, or random-digit dialing methods for survey-based research as well as for recruitment into clinical trials [15]. EHRs provide an avenue for screening and then directly targeting recruitment at potentially eligible patients for research studies [4,15]. Recruitment of patients from EHRs has the potential to be more inclusive by including patients who might be sicker and those who are undergoing care in real-world health care systems [5,16]. However, no method of recruitment is without the potential for selection biases. With EHR-based recruitment, there are concerns about external validity or generalizability to nonpatient or health care populations, particularly for prevention-oriented studies [17]. In addition, patients with reduced access to care or those who are not regularly followed in the health care setting may be less likely to be approached in person (ie, at the time of a clinic visit) or to receive recruitment messages when they do not have an email address or access the patient portal. While 90% of US adults now use the internet, older adults and Black citizens still show lower rates of internet access [18], but technology usage rates are increasing [19]. A recent study by Walker and colleagues [18] showed that fewer Black patients and patients over the age of 70 had access to an inpatient portal. Notably, in our study, which used a combination of recruitment strategies, approximately 9% of enrolled participants identified as Black compared with 11% among patients identified as potentially eligible. In fact, researchers could leverage the EHR to specifically target patient populations by demographics or medical diagnosis (eg, by race, ethnicity, age, or rare health conditions [20,21]).

Despite the increasing need for effective and less staff-intensive methods of recruitment for clinical trials and surveys, few studies have compared recruitment strategies [22,23]. One study reported and compared response rates from various recruitment methods (postal survey, postal invitation to complete an internet survey, and postal invitation for a telephone survey) for an environmental survey but did not include EHR-based recruitment methods [23]. The study showed the highest response rate (30%) for the telephone survey [23]. The EHR provides a new way to identify potential participants for research studies by applying electronic eligibility criteria to sometimes very large pools of potentially eligible patients [24].

A systematic review by Lai and Afseth [25] assessed the effectiveness and efficiency of EHR-based recruitment methods. They identified 13 articles, of which 11 reported recruitment efficiency and most used alerts sent directly to physicians or staff to notify of participant eligibility [25]. EHRs have multiple functionalities to support recruitment [26]. In our study, we utilized the EHR for several purposes: to generate lists of potentially eligible patients using a computable phenotype, to obtain postal mailing addresses and email addresses, and to send recruitment invitations using the secure patient portal at 2 sites. Some academic research centers have designed patient portal recruitment services to enable, but also limit, recruitment using the portal for certain approved studies [1,21,27,28]. In a 2019 single-institution study that included 13 separate EHR-based recruitment strategies using the patient portal recruitment service, the average response rate for patient portal messages was 2.9%, which was the same as our enrollment rate for both email and patient portal recruitment [21]. Interestingly, we offered no compensation to patients to enroll in our study, yet the studies reviewed by Miller et al [21] did offer compensation.

Although the computable phenotype enabled our teams to identify a very large number of patients who were potentially eligible, each health system designed its own outreach methods to patients by targeting specific clinical sites, thereby greatly reducing the potential number of patients that could have been contacted about this research opportunity. Importantly, institutional review boards at each site prohibited the “cold calling” of patients (ie, directly contacting potential research participants based on prior knowledge of the patients’ health information in the absence of a treatment or clinical relationship [29]). Therefore, each site partnered with primary care and bariatric surgery providers, who approached the patients about the study first, either with their signature on the recruitment invitation or through an in-person introduction. However, a recent landscape analysis by McHugh and colleagues [29] highlighted that this universally applied “ban on cold calling” could impose a gatekeeping function and potentially reduce patient autonomy, decreasing access to research and risking the introduction of selection bias into research studies. They suggested alternative approaches to ensuring patient privacy [29], with the goal of broadening access to health research participation [15].

We identified several limitations of this study. First, this was a 5-center, multisite study and the deployed recruitment strategies were dependent on the norms for research recruitment at each site, with the implication that not all samples were directly comparable by site or by strategy. For example, 3 sites did not allow research recruitment using the EHR patient portal and 1 site did not have access to email addresses in the EHRs. Because of the differences in clinical populations’ demographics between sites, there was high correlation between the site’s choice of recruitment methods and the patients who were recruited, limiting our ability to draw conclusions about whether specific recruitment modalities are more (or less) effective for specific racial/ethnic groups or different age categories. Second, the overall response rate was low; however, it is very comparable to response rates from other studies that relied on EHR-based or email-based recruitment. We were able to assess for selection biases by comparing those patients who were deemed eligible with those who enrolled in the study. We showed lower uptake among Latinx patients, but this was in part because the consent and survey were limited to participants able to read in English. Third, although this study was low burden for participants, as they consented to having the research team review medical records and complete a 20-minute online survey, we did not offer any compensation for their time, which could have limited enrollment. Therefore, we are not able to draw conclusions about how effective our recruitment strategies would be for more intensive studies, for studies not using online data collection, or for studies offering participant incentives. Fourth, we were unable to estimate cost or cost-effectiveness of the recruitment strategies or describe in detail the staffing time or corresponding resources involved for each health system, such as the effort for the health informatics team to support EHR-based research or the time for research staff to conduct in-person clinic-based recruitment. Fifth, recruitment for this study occurred approximately 4 years ago (2015-2016) and it is possible that response rates and methods for electronic recruitment have improved over time. Sixth, the 5 health systems were all based in the mid-Atlantic region of the United States, which could limit the generalizability of these findings to other regions of the country and outside the United States.

The major implication of our study is to inform the selection of recruitment strategies for the design of future cohort studies, utilizing the capabilities of the modern-day EHR system. We anticipate that other researchers could find this information useful in the design of their recruitment strategies and to estimate the expected yields from “low touch” strategies that require less personnel contact with potential participants. As health systems and institutional review boards become more comfortable and familiar with EHR-based recruitment, it will be possible to achieve greater consistency between the recruitment processes and even sample selection to reduce biases across systems. Ultimately, to facilitate EHR-based research across multiple settings (eg, inpatient and ambulatory), large health systems will need to invest in infrastructure to support and link smaller clinical centers and subsidize their use of a common EHR. An example of a successful academic and community-based network is the OCHIN network, which provides a research infrastructure using EHR data from a national network of smaller community health centers [30].

In conclusion, we recruited a total of 1085 patients from primary care and bariatric surgery clinics to complete a survey and participate in an EHR-based cohort study using 4 recruitment methods. The greatest recruitment yield was achieved using the email-based method, but the greatest efficiency resulted from in-person recruitment. Implementation of low-resource recruitment approaches has important implications for future patient-centered studies in health system settings.