Eligible patients from the University of California, San Francisco (UCSF) Medical Center were invited to join the Health eHeart Study [12] via an email recruitment campaign carried out from August 2015 to February 2016. The Health eHeart Study is a worldwide digital cardiovascular health–focused electronic cohort (eCohort) coordinated at the UCSF. Participation in the Health eHeart Study is open to adults (age ≥18 years) who understand English and have an active email. Health eHeart participants are recruited through a variety of modes, including clinic visits, word-of-mouth, lay press, social media, promotional events, and email. After web-based registration (name, date of birth, email, and password) and consent, participants are prompted to complete web-based surveys about their basic and social demographics, medical history, family history, activity and well-being, habits and lifestyle, mental health, diet and nutrition, and technology use. Participants also have the option to “connect” mobile health devices and apps to contribute additional data to the study. The Health eHeart Study’s design and procedures (eg, user interfaces, incentives, technical support, and reminder schedule) were not preferentially altered for the email recruitment campaign or subgroups of the invited patients.

The UCSF institutional review board approved both the Health eHeart Study and the analysis of this digital recruitment campaign (#15-18180). Ethics approval covers secondary data analyses without additional consent in accordance with institutional guidelines. Protective measures (eg, encryption) were carried out to safeguard all study data.

Patients are defined as those who have a patient record within the UCSF electronic health record (EHR) system. Living UCSF patients who are 18 years of age or older, with a documented email address within the EHR, and with English recorded as their preferred language were sent an email invitation (Multimedia Appendix 1). The email invitation included a short description about the study mission and a “call-to-action” button for patients to register. The button launched a browser with a patient-specific URL including a linkage identifier that enabled the Health eHeart Study to link their Health eHeart account to the specific UCSF patient receiving the email. Emails were designed and scheduled through Mailchimp (Intuit), an email marketing platform that also provides recipient delivery and engagement metrics. The email campaign was segmented into 15 initial “waves” of recruitment (Multimedia Appendix 2). Patients for each wave were randomly selected. Different waves also received varying subject line messaging and delivery days and times to maximize variation in the recruitment outreach strategy. Patients from waves 1 to 14 were sent 1 follow-up email invitation if they remained unregistered (excluding unsubscribes and hard bounces) for at least 2 weeks after the initial invitation.

We conducted a cohort analysis of all invited UCSF patients to identify predictors of enrollment. Data were extracted from the UCSF EHR system (Epic Systems Corporation) between September 2019 and December 2019, for sociodemographic, clinical diagnoses, health care use, and insurance coverage information at the time of the patient’s email invitation date. We limited the analysis to contacted patients, defined as those sent an email via Mailchimp that was not rejected by the patient’s email server.

The primary outcome of interest was enrollment, defined by account registration and consent into the Health eHeart Study, among contacted patients. In the Health eHeart Study, enrollment occurs in 2 crucial steps: registration and consent. We also looked at these 2 steps separately. We defined registered patient participants as those who set up a Health eHeart Study account with a name, date of birth, email, and password. We defined consented patient participants as registered patients who indicated their willingness to participate, after being shown the consent form, by clicking on an “I want to participate” button.

Patient-level sociodemographic variables from the EHR include (1) age (<30, 30-39, 40-49, 50-59, 60-69, 70-79, ≥80 years); (2) sex (male and female); (3) race or ethnicity (non-Hispanic White, non-Hispanic Black, non-Hispanic Asian or Asian and Pacific Islander [API], Hispanic or Latino, other, multiracial, and unknown or declined to state); (4) insurance (commercial, Medicaid, Medicare, other, and unknown or declined to state); and (5) marital status (married or partnered, not married or partnered, and unknown or declined to state). Insurance status serves as a proxy for individual socioeconomic status (SES) since income levels cannot be ascertained from the EHR. Insurance statuses of Medicaid or unknown or declined to state were used as indicators of lower SES.

We derived the following patient-level clinical variables using EHR data extracted from UCSF’s clinical data warehouse: (1) Charlson Comorbidity Index (CCI) score (0, 1, 2, and 3+); (2) diagnosis of medical condition (myocardial infarction, congestive heart failure, peripheral vascular disease, cerebrovascular disease, dementia, chronic pulmonary disease, rheumatologic disease, peptic ulcer disease, diabetes, liver disease (mild, moderate, or severe), paraplegia or hemiplegia, cancer (any type or metastatic solid tumor), renal disease, HIV/AIDS); and (3) UCSF recent patient status (inactive and active). Patients are defined as “active” if they had 1 or more health care encounters at UCSF 6 months prior to the initial email invitation. The CCI score is a validated standardized measure of overall comorbidity burden and generates a weighted index based on the relative risks of 1-year mortality or “the number and seriousness” of 17 comorbid diseases [13,14]. CCI scores were calculated at the time of the initial contact email and therefore represent each patient’s comorbidity status at the time of recruitment.

For recruitment-related variables, we also adjusted for the number of recruitment emails received (1 vs 2). Patients in wave 15 were only delivered 1 (initial) email, while those in waves 1 to 14 had the potential to be delivered up to 2 (initial and follow-up) email invitations.

Descriptive statistics were performed to characterize sociodemographic and clinical distributions for (1) all contacted patients, (2) registration status among contacted patients, and (3) consent status among registered patients. Patient characteristics were compared between (1) registered versus did not register and (2) consented versus did not consent using bivariate analysis. Categorical variables are presented as frequencies (percentages) and compared using chi-square tests. Continuous variables with skewed distributions are presented as medians (IQR). We used logistic regression to simultaneously adjust for all patient- and email campaign–related variables. We first performed a univariate (unadjusted) analysis for each variable (Multimedia Appendix 3), then performed a multivariate (adjusted) analysis that included all variables. The results are summarized as unadjusted odds ratios or adjusted odds ratios (aORs), respectively, with 95% CIs. All statistical analyses were conducted using STATA (version 16.0; StataCorp), and P values <.05 were considered statistically significant.

Figure 1 shows the flow of recruited patients from invitation to registration and web-based informed consent. An EHR data search generated a recruitment list of 210,385 eligible patients. Due to invalid or incomplete records, 3402 patients were excluded from recruitment or analysis. An initial recruitment email was sent in 15 waves to a total of 206,983 patients, where approximately 6.5% (n=13,448) returned a bounced email, resulting in an initial contact rate of 93.5% (n=193,535). The primary barrier to enrollment was registration (Figure 1). Among those initially contacted, only 2.6% (5101/193,535) registered, of which 84.5% (4332/5101) consented.

A follow-up recruitment email was sent to a total of 116,965 patients from waves 1 to 14, who were initially contacted but did not register or whose initial email bounced. Approximately 1.3% (n=1554) returned a bounced email, resulting in a follow-up contact rate of 98.7% (n=115,411). Among those recontacted, 1.7% (1911/115,411) registered, of which 81.2% (1567/1911) consented.

Overall, the email recruitment campaign delivered at least 1 email invitation to 93.5% (193,606/206,983) of all recruited patients, which yielded a 3.6% (7012/193,606) registration rate among contacted patients and an 84.1% (5899/7012) consent rate among registered patients. The overall enrollment rate for registration and consent was 3.6% (7012/193,606) and 3% (5899/193,606) among ever-contacted participants, respectively.

Patient characteristics for all contacted patients, by their registration and consent status, are reported in Table 1. In terms of sociodemographic characteristics, the median (IQR) age of 193,606 contacted patients was 48.5 (36.0-62.6) years, with the majority being non-Hispanic White (n=115,418, 59.6%), women (n=113,455, 58.6%) with commercial insurance (n=107,972, 55.8%), married or partnered (n=100,071, 51.7%). Approximately half (n=96,516, 49.9%) were considered active patients of UCSF and 62.1% (120,149/193,606) were part of a wave with scheduled follow-up emails. In terms of clinical characteristics, while the majority (124,861/193,606, 64.5%) of contacted patients had a CCI score of 0 (ie, no comorbidity burden or severity), the most reported medical conditions among contacted patients were cancer (28,677/193,606, 14.8%) and chronic pulmonary disease (18,771/193,606, 9.7%). Table 1 also shows how sociodemographic and clinical characteristics significantly differed by registration status among all contacted patients and consent status among registered patients.

Figure 2 presents the results from 2 adjusted multivariate logistic regression models. The first model analyzes independent predictors of registration among contacted participants; the second model analyzes predictors of consent among registered participants. All variables were significant in the first registration logistic model, controlling for all other variables in the equation, except for marital status and certain medical conditions.

Higher age groups were associated with greater odds of registration; compared to those <30 years, the odds of registration was significantly higher for every decade increase in age from 40-49 (aOR 1.79, 95% CI 1.58-2.02), 50-59 (aOR 2.62, 95% CI 2.33-2.96), and 60-69 (aOR 3.78, 95% CI 3.35-4.26) to 70-79 (aOR 4.59, 95% CI 4.00-5.25), before waning slightly for those ≥80 years (aOR 3.30, 95% CI 2.77-3.93). Relative to men, women (aOR 1.35, 95% CI 1.28-1.42) had greater odds of registration. Compared to non-Hispanic White patients, the odds of registration were lower among patients who were non-Hispanic Black (aOR 0.48, 95% CI 0.41-0.57), non-Hispanic Asian or API (aOR 0.53, 95% CI 0.48-0.59), Hispanic or Latino (aOR 0.87, 95% CI 0.78-0.96), other (aOR 0.73, 95% CI 0.65-0.82), or unknown or declined to state (aOR 0.74, 95% CI 0.67-0.82) race or ethnicity. Relative to those with commercial insurance, those with Medicaid (aOR 0.81, 95% CI 0.72-0.91) or unknown or declined to state (aOR 0.75, 95% CI 0.68-0.82) insurance had lower odds of registration.

Those from waves 1 to 14 (aOR 1.80, 95% CI 1.70-1.90), which included a follow-up email, had greater odds of registration than those from wave 15, where there was no follow-up email. Active UCSF patients (aOR 1.58, 95% CI 1.49-1.67) had greater odds of registration compared to inactive patients. Compared to those with a CCI score of 0, patients with some comorbidity were more likely to register (aOR 1.11, 95% CI 1.01-1.22 for a score of 1 or aOR 1.18, 95% CI 1.04-1.35 for a score of 2). A diagnosis of congestive heart failure (aOR 1.28, 95% CI 1.10-1.50) was independently associated with greater odds of registration, while a diagnosis of dementia (aOR 0.44, 95% CI 0.31-0.62), chronic pulmonary disease (aOR 0.89, 95% CI 0.81-0.98), moderate or severe liver disease (aOR 0.62, 95% CI 0.45-0.86), paraplegia or hemiplegia (aOR 0.54, 95% CI 0.35-0.83), renal disease (aOR 0.81, 95% CI 0.70-0.94), or any type of cancer (aOR 0.80, 95% CI 0.70-0.90) were associated with lower odds of registration after adjustment for overall comorbidity and age (both strongly associated with specific conditions).

Only sociodemographic variables (except for marital status) and 1 medical condition were significantly associated with consent among registered patients after adjusting for all other variables in the equation.

Higher age groups were associated with lower odds of consent; compared to those <30 years, the odds of consent significantly decreased in age groups from 30-39 (aOR 0.66, 95% CI 0.44-1.00), 40-49 (aOR 0.57, 95% CI 0.38-0.85), 50-59 (aOR 0.42, 95% CI 0.29-0.62), 60-69 (aOR 0.44, 95% CI 0.30-0.64), 70-79 (aOR 0.44, 95% CI 0.29-0.66) to ≥80 (aOR 0.36, 95% CI 0.22-0.58) years. Relative to men, women had (aOR 0.68, 95% CI 0.59-0.79) lower odds of consent. Compared to non-Hispanic White patients, the odds of consent were lower among patients who were non-Hispanic Black (aOR 0.51, 95% CI 0.35-0.75), non-Hispanic Asian or API (aOR 0.59, 95% CI 0.46-0.75), Hispanic or Latino (aOR 0.52, 95% CI 0.40-0.68), other (aOR 0.58, 95% CI 0.43-0.79), or unknown or declined to state (aOR 0.75, 95% CI 0.57-1.00) race or ethnicity. Relative to those with commercial insurance, those with Medicaid (aOR 0.65, 95% CI 0.48-0.87) had lower odds of consent. A diagnosis of peripheral vascular disease (aOR 0.65, 95% CI 0.44-0.94) was also associated with lower odds of consent.

The effect sizes for both models—registration and consent—are displayed alongside each other to show the scale of differences between them. Unadjusted models are included in Multimedia Appendix 3.

We set out to describe the sociodemographic and clinical characteristics that predict enrollment in the Health eHeart Study with an email-based recruitment campaign to patients in a health system. Initial engagement rates (registration among all contacted patients) were generally low (7012/193,606, 3.6%), with higher rates among patients who are older, women, non-Hispanic White, active patients with commercial insurance or Medicare, with a higher comorbidity burden, with congestive heart failure, and randomized to receive up to 2 recruitment emails. Patterns of subsequent consent after initial registration were somewhat different (with inverse trends by age and gender), but enrollment was strongly driven by initial engagement since most initially engaged patients end up consenting to join the study (5899/7012, 84.1%). Overall, enrollment was also driven more strongly by sociodemographic than clinical factors.

Email is an extremely efficient means of recruiting participants from a large list into the Health eHeart Study, a digital health study. The email recruitment campaign required relatively low operational effort from the research team and cost less than US $1000 to carry out through Mailchimp. The campaign led to 7012 patients registered, 5899 of whom consented, into the Health eHeart Study within 6 months. With access to a massive email list, the campaign was able to recruit a sufficient absolute number of total participants for most clinical trial needs. However, it also resulted in a lower proportion of patients who are racial or ethnic minorities and of lower SES. Despite email showing some advantages with recruitment, the underrepresentation of racial or ethnic minorities and those of lower SES was found to persist even at these early enrollment stages of a digital research study.

The extent to which older patients registered into the Health eHeart Study compared to their younger counterparts supports the notion that remote recruitment via email can be particularly effective in enrolling older people in the right context. This can be partially attributed to the study’s disease focus and institutional affiliation; heart health may inherently be more interesting to older patients seeking cardiovascular care at a research hospital compared to older people exploring a public forum. This finding contrasts with literature showing that those who enrolled in a research study through various digital channels tend to veer younger relative to the recruited population [15-18]. Historically, older adults are underrepresented in clinical trials, but this may be attributed to ageism and the associated biases and assumptions that hinder their recruitment [10,11,19]. One assumption is that older adults are more apprehensive toward technology; hence their underrepresentation may persist in digital health studies, but studies have shown that that barrier is smaller than once imagined [17,20]. The COVID-19 pandemic also accelerated older adults’ adoption of digital health technologies, furthering the need to update recruitment approaches to maximize their inclusion in digital research [19].

Female patients, to a lesser effect, were also more likely to participate in the Health eHeart Study compared to male patients. The underrepresentation of women in conventional studies has been partially attributed to structural factors, such as time demands and scheduling, study type (eg, randomized trial), financial incentives, seasonality, and clinical environment, among many other reasons [21-23]. In contrast, women were consistently reported to be more likely to participate in digital research studies [3,24], which can help bypass structural barriers to research participation. The findings from the email campaign further support the reversal in gender representation in digital research studies.

Consistent with existing trends [25,26], patients who identified as racial or ethnic minorities were noticeably less inclined to participate in the Health eHeart Study compared to non-Hispanic White patients at both the registration and consent steps. Even though racial or ethnic minorities have indicated to be as willing to participate in research as White patients, there continue to be disparities in their representation in research [27-29]. This can be attributed to multilevel barriers to participation, which can range from individual (eg, distrust, lack of study awareness), interpersonal (eg, health professional biases), and systemic factors (eg, inequities in health care access and research inclusion) [26,29-32]. In our study, for example, the UCSF EHR produced a significantly lower proportion of eligible racial or ethnic minorities (even though our eligibility criteria were especially broad) to contact and invite into the Health eHeart Study.

Furthermore, patients with Medicaid or “unknown or declined to state” insurance were less likely to register compared to those with commercial insurance. However, there was a significant drop-off at the consent stage observed among patients with Medicaid insurance. Systemic factors such as lack of adequate insurance coverage and internet access are especially burdensome for racial or ethnic minorities and those with lower SES. Patients’ state of insurance coverage may be an indicator of their digital research participation [23,27-29,33]. In addition, the “digital divide” or disparity in internet access (ie, a growing share of Americans that are low-income, Black, or Latino are becoming smartphone-only internet users) also hinder the participation of underrepresented groups in digital health studies [34-36]. The recruitment of underrepresented groups in research will require higher-touch approaches with significant investment in relationship building to overcome persistent barriers.

Most medical conditions that we studied, and medical comorbidity burden in general, did not appear to be obstacles to enrollment after adjustment for other factors. Patients with a low comorbidity burden, either CCI score of 1 or 2, were more likely to register compared to those with no relevant comorbidities. This contrasts with the classic “healthy volunteer” effect that suggests those who join clinical trials or studies are healthier than those who decide not to participate [37,38]. Unsurprisingly, due to the heart health focus of the study, patients with congestive heart failure were more likely to register compared to those without the condition. Similarly, patients with dementia and several other conditions were less likely to enroll after adjustment for comorbidity burden. There is limited literature on the clinical makeup of participants who are remotely recruited to join digital health studies, much less the entire population reached by their remote recruitment efforts.

Operational considerations of a recruitment plan are also important for maximizing study enrollment [39]. Incorporating follow-up schemas in recruitment campaigns can help research teams boost their chances of reaching and enrolling people from a given contact list. Active patients are shown to be more receptive to email invitations from their current health care system. Narrowing recruitment to a presumably more engaged or readily accessible sample can help optimize enrollment yields and conserve limited study resources.

The findings from the study reveal a more nuanced understanding of using email as a digital health study recruitment tool. While there is no definitive recruitment strategy that fully addresses diversity shortcomings in digital health studies, the email campaign has demonstrated effective recruitment of certain underrepresented groups (eg, older adults and women) in contrast to prior studies that included email [16,40]. The study also provides extensive participant characterization in contrast to other studies of digital recruitment tool effectiveness [15,41,42]. While our findings are most pertinent to email recruitment, other digital channels such as SMS text message, phone calls, or web-based chat may have differential (and underexplored) potential in recruiting diverse populations. Multichannel recruitment can help better reach underrepresented groups, but it also requires procuring additional sensitive information about the recruited population. Future email campaigns may need to carry out participant-centered research to help optimize trust and cocreate messaging, including in partnership with community organizations, patient advocacy groups, or charity organizations. To adapt recruitment strategies to be more inclusive and equitable, study teams need to understand the fundamental needs of target recruited populations (ie, those considered underrepresented or vulnerable populations in research, have rare disease conditions, and people from low- and middle-income countries) and address enrollment barriers (eg, lessen time and economic burden of study procedures and provide recruitment materials in various languages). Once email campaign procedures and messaging are deemed appropriate and inclusive, these partnering groups can also help further expand the reach of study recruitment efforts by engaging members from their respective email listservs.

Finally, recruitment is only the beginning of the digital research study life cycle. Retention and engagement of enrolled underrepresented participants throughout a study’s life course are equally important for generating equitable health discoveries and benefits. Digital research studies need to be inclusive by design and supported in ongoing initiatives to achieve equity at all stages of participation. For the next generation of “precision population health” eCohort studies, such as the NIH’s one-million-person All of Us Research Program [43], recruitment strategies will need to be as dynamic as the diverse populations they hope to reach to break through established barriers to digital health study participation.

This study had important limitations. First, the way in which patients were included in our recruitment campaign may be a source of selection bias. Despite Health eHeart’s broad eligibility criteria, UCSF patients with documented email addresses were also those with MyChart patient portal accounts, for which there are known baseline differences across patient populations. There are also inherent differences among patients who list English as their preferred language compared to those who do not. Second, recruitment was restricted to UCSF patients who are not representative of the regional or general population. It has also been reported that “university-led” studies have higher levels of participation [18]. Thus, enrollment disparity patterns observed in our analysis may vary greatly if the email recruitment campaign were deployed to the general population. Third, while the subject line messaging and delivery days and times were varied as part of the recruitment outreach strategy, the content of the email remained constant. For example, the image used in the email might have been varied with consideration of patient-physician concordance [44]. It is recommended that subsequent email initiatives will need to further tailor and fine-tune recruitment materials to better resonate with the recruited population. Fourth, our study did not consider the potential role of interactions in our analyses. Finally, although our email recruitment campaign occurred in 2015-2016 and digital research participation patterns may have changed since then, there remain limited comprehensive evaluations on the effectiveness of email for digital health study recruitment. This study offers important insights into the potential returns on participation when study teams invest resources in using email for recruitment. However, it remains to be seen whether the effect of email recruitment will be further diluted considering the COVID-19 pandemic [45-48] and amidst a digital landscape in excess of outlets vying for one’s attention.

The purpose of this study was to understand who volunteers to participate in a digital health study because of an exclusive and large-scale remote email recruitment campaign. Contrary to other mass media campaign evaluations, our study was able to characterize the sample of patients who did and did not enroll, whereas previous evaluations had limited to no information about the latter group. Overall, the findings showed that enrollment was driven more strongly by sociodemographic than clinical factors. Email is also an extremely efficient means of recruiting participants from a large list into the Health eHeart Study. Despite some improvements in representation, the formulation of truly diverse studies will require additional resources and strategies to overcome remaining participation barriers.