This was a retrospective cohort study conducted at the UCHealth health system, which is a large regional health system in Colorado and areas of Wyoming and Nebraska. UCHealth is comprised of 14 hospitals, 900 clinics, and over 6000 physicians who serve academic, rural, suburban, and community settings. The entire system is serviced by a single instance of the Epic EHR software program (Epic Systems, Verona, WI, United States).

This protocol, #22-1154, was approved by the University of Colorado Multiple Institutional Review Board. This was a secondary analysis of deidentified patient data.

We collected data on eligible patients from Health Data Compass, our institutional enterprise data warehouse that extracts, integrates, and delivers data from the EHR for patients within the UCHealth system. Data from Health Data Compass also includes data from the public health department for Colorado and US census data, which were used to supplement EHR data related to death dates and location of residence (ie, rural, urban). The primary outcome of interest was the odds of at least one virtual visit, defined as a synchronous meeting between a clinician and patient, during the 1-year evaluation period from September 1, 2021, to August 31, 2022.

Patients meeting the following criteria were included in our study: alive as of September 1, 2021 with a history of an ejection fraction (EF) ≤40%, and at least one virtual or in-person outpatient visit to a primary care or cardiology clinician in the UCHealth system in the prior year (between September 1, 2020 and August 30, 2021).

Cardiology and primary care clinicians were defined based on department specialty. Among these specialties, outpatient visit types were limited to those that were used by physicians and advanced practice clinicians (eg, excluded nurse or lab-only visit types).

The primary outcome of interest was the odds of at least one virtual visit versus no virtual visit during the 1-year evaluation period from September 1, 2021, through August 31, 2022. This period was chosen to reflect the most recent data available. Secondary outcomes included the frequency of primary care, cardiology, emergency department, and hospitalization encounters stratified by in-person versus virtual visits.

There is a large number of patient characteristics and equity covariates to consider; key features were selected by the study team who represent expertise in advanced heart failure, geriatrics, primary care, health equity, digital divide, clinical pharmacy, and hospital medicine. Features were selected based on univariate statistical significance, availability, completeness, and reliability of data collected in structured data fields, as well as if they were recognized as characteristics that influence how patients with HFrEF receive care.

Age and other patient characteristics were assessed at the start of the evaluation period. The National Center for Health Statistics (NCHS) Urban/Rural County designation scheme [11] was used to assign urban/rural status for each patient. Patients who were missing NCHS data but had a valid zip code within the state of Colorado were assigned urban/rural status based on their Colorado county of residence. Patients missing both NCHS and zip code data, or missing NCHS and living outside of Colorado, were assigned an urban/rural status of unknown. Due to large amounts of missing social determinants of health (SDoH) data (eg, transportation, food insecurity, homelessness), the Social Vulnerability Index (SVI) decimal value [12], based on zip code, was used to evaluate patient-level SDoH. The SVI is a widely recognized tool for measuring community-level vulnerabilities to environmental hazards. It integrates multiple socioeconomic and demographic variables to quantify social vulnerability. Determinants of the digital divide included data routinely collected within the EHR that have previously been found to be associated with the digital divide and were as follows: patient portal status, cell phone on file, and email on file in the EHR.

Descriptive statistics were used to evaluate baseline patient demographics and clinical use data and outcomes. Categorical data were presented using counts and percentages, while discrete and continuous data points were presented using means, SDs, medians, minimums, maximums, and ranges where appropriate. Univariate analyses were performed both with virtual visits as a variable (received or did not receive) using the chi-square test for association and as a discrete outcome using the Wilcoxon rank-sum test to capture potentially important predictor variables that could influence the use or frequency of using virtual visits. Univariate comparisons for SVI, in-person primary care provider (PCP)/cardiology visits, and the most recent EF used simple logistic regression to model individual odds ratios (ORs).

For the primary outcome, a multivariable logistic regression model was used to model the odds of at least one virtual visit with a primary care or cardiology clinician during the study follow-up period. Covariates for this model were first chosen through clinical reasoning. Additional covariates exhibiting univariate significance were subsequently added to the adjusted model to evaluate their individual impact, as well as the impact on clinical covariates previously included. This multivariable model was also assessed for correlation between parameter estimates of predictor variables using a correlation matrix. Secondarily, in exploratory analyses, a negative binomial regression model was used to assess the incident rate of virtual visits during the evaluation period. The same patient characteristics of interest were adjusted for in this multivariable model as the adjusted logistic model mentioned above. These specific variables were estimated for their relationship with the rate of the outcome. Multiple variables were presented for evaluation of portal use (active patient portal status and portal ever registered), as well as a phone number listed on the patient file (home phone only, cell phone). Due to multicollinearity and incomplete data, portal registration and home phone only were excluded from all analyses. Patients with missing data for covariates were excluded from all multivariable analyses. Additional cross-tabulations and frequencies are presented in Multimedia Appendix 1 showing digital divide covariates stratified by demographic variables of interest. All statistical analyses were performed using SAS software (SAS Institute Inc).

A total of 8481 patients were included in the analysis. This cohort reflects a largely urban population, where 7132 (84.1%) patients were categorized as residing in an urban location type based on zip code. Mean age was 65.90 years (SD 15.1) with 2400 (28.3%) participants older than 75 years. Included were 2807 (33.1%) female patients, 6608 (77.9%) White patients, 7356 (86.7%) non-Hispanic patients, 4711 (55.6%) patients on Medicare, and 3161 (37.3%) patients who had a most recent EF of ≤40%. Most patients (8035/8481, 94.7%) preferred English, and 396 (4.7%) of patients had ever needed or used an interpreter.

Variables that had a high percentage of missingness included employment status with 97.2% missing, and SDoH data including transportation, food insecurity, living conditions, homelessness, and housing status where <1% of patients had reliably reported data. Table 1 contains a description of the population at baseline.

The large majority of patients had no cardiology (7938/8481, 93.6%) or primary care (7955/8481, 93.8%) virtual visits (Table 2). Of those that did receive at least one virtual visit, the mean number of visits was 1.4 (range 1‐10) for cardiology and 1.6 (range 1‐12) for primary care. In contrast, 2001 (23.6%) and 5957 (70.2%) of patients had no in-person visits in cardiology and primary care, respectively. Of those that did receive at least one in-person visit, the mean number of in-person visits was 2.8 (range 1‐38) for cardiology and 3.0 (range 1‐22) for primary care. We also explored how patients receive care over a combination of different visit types for the cohort as a whole and across multiple subgroups of patient characteristics to assess for differences in health care use. Table S1 in Multimedia Appendix 1 displays these data.

Table 3 lists the patient characteristics that were evaluated in the final model. In univariate comparisons (Table 3), the only significant findings were variables related to digital literacy, patient cell phone in EHR (OR 2.85, 95% CI 2.1-3.8; P<.001), patient email address in EHR (OR 9.29, 95% CI 5.6-15.5; P<.001), active patient portal status (OR 2.80, 95% CI 2.3-3.4; P<.001) and patients who had in-person primary care or cardiology visits had a higher mean number of virtual visits (6.32, SD 5.90) compared with those who did not (3.3, SD 3.82) (P<.001).

The mean number of virtual visits was 0.24 (SD 0.87) for those who had a cell phone listed in the EHR, compared with a mean of 0.07 (SD 0.40) virtual visits for those who did not (P<.001). The mean number of virtual visits was 0.24 (SD 0.90) for those who had an email address listed, compared with a mean of 0.02 (SD 0.15) virtual visits for those who did not (P<.001). The mean number of virtual visits was 0.11 (SD 0.58) for those who had an inactive patient portal status compared with a mean of 0.27 (SD 0.91) virtual visits for those who had an active patient portal status (P<.001).

Of the 8481 patients included in the study, 7837 (92.41%) had complete data with no missing covariate information. After adjusting for other covariates in the model, the primary multivariable logistic model showed significance in at least one virtual visit for a patient having a cell phone listed in the chart, email address on file, active patient portal status, and in-person visits with a primary care or cardiology clinician. Patients with a cell phone listed were at 68% higher odds of the outcome compared with patients without a cell phone listed (OR 1.68, 95% CI 1.23‐2.29). Patients with an active patient portal status were at 62% higher odds than their counterparts with an inactive or other patient portal status (OR 1.63, 95% CI 1.32‐2.00). Patients with an email address on file were at the highest odds of the outcome, with their odds being 5.18 times greater than patients without an email address on file (OR 5.18, 95% CI 2.96‐9.05). Last, for each in-person visit a patient had with a primary care or cardiology clinician, their odds of having the outcome of a virtual visit increased by 12.5% (OR 1.13, 95% CI 1.11‐1.14). Assessment of the correlation matrix did not exhibit high rates of correlation between parameter estimates of covariates in the model, with the highest correlation being 25% between active patient portal status and an email address on file.

Similar results were found when modeling the incident rate of virtual visits. The incident rate of a virtual visit was 7.36 times greater for patients with an email address on file compared with patients without an email address on file (95% CI 4.21‐12.87). Additionally, the incident rate was 60% higher for patients with an active patient portal status (95% CI 1.30‐1.97), and 87% higher for patients with a cell phone on file (95% CI 1.36‐2.57) compared with patients without an active patient portal status, and without a cell phone in file, respectively. Last, each in-person visit with a primary care or cardiology clinician was associated with a 14.4% increase in the rate of the outcome of a virtual visit (95% CI 1.12‐1.67). No other patient characteristics were significantly associated with the outcome of virtual visits. Table S2 in Multimedia Appendix 1 displays the parameter estimates for both the logistic and negative binomial models.

To further explore differences in access to digital tools, we cross-tabulated digital divide indicators including cell phone in the EHR, Portal Status, and email in the EHR by patient characteristics and did not see evidence of differences between the age, sex, race, ethnicity, rurality, use of an interpreter, insurance status, EF, or SVI (Table S3 in Multimedia Appendix 1).

In this post-COVID-19 pandemic cohort of patients with HFrEF, only a small number of patients with HFrEF had virtual visits with most outpatient visits occurring in person and more often in cardiology clinics than primary care. We also found that indicators of digital access such as having an email address on file or having an active EHR patient portal status were associated with having virtual visits. Given the relatively small number of minoritized patients within this cohort and the significant missingness of SDoH data collected, we were unable to fully evaluate for differences in patients receiving virtual care based on demographic and SDoH factors.

To our knowledge, this study is one of the first to offer estimates of virtual visits in patients with heart failure postpandemic and demonstrates a much lower use of virtual visits than during the pandemic [13]. Literature describing virtual visits for heart failure documents that while they were considered necessary during the pandemic [10] and can alleviate some burden of frequent visits, they are felt to have limitations in the care of heart failure particularly with regard to volume status assessment [14], which aligns with our findings of few virtual visits.

Although there is some evidence that virtual interventions can improve heart failure management [15], there is less evidence specifically on the impact of virtual synchronous visits alone for heart failure. For instance, telehealth interventions that incorporate multiple physiological measures tend to be more efficacious than interventions that rely on video visits alone [3]. In the case of heart failure management, it generally requires multiple physiologic measures to make informed decisions; thus, standard virtual visits without such telehealth interventions may not be sufficient. While there is currently limited evidence supporting the use of virtual care as a replacement for in-person care for heart failure management, the potential for virtual care augmented with telehealth interventions to improve outcomes and quality of care remains compelling [16].

Our finding that HFrEF patients are seen more often by cardiology than primary care is to our knowledge a novel contribution to the US heart failure literature. While estimates can be found in Swedish cohorts with 44% of patients managed by non-cardiologists [17], to our knowledge, no recent estimates have been published in US populations. That said, it seems likely that the frequency of follow-up with cardiology relative to primary care may vary substantially between health systems that serve communities with different payer mixes and access to specialists. Thus, these findings may have limited generalizability and should be evaluated in other health systems.

Another important finding was the large amount of missingness of the SDoH data including food insecurity, transportation, education, homelessness, and employment. This is a notable finding as these measures are known to predict health outcomes in many patient populations [18]. Although important, it is not surprising as the incompleteness of collecting SDoH data within the EHR is a well-recognized problem [19-21]. Without these data, needs such as housing and food insecurity cannot be addressed at the point of care, and additionally, disparities in care delivery and outcomes cannot be accurately measured at the health system level. The American Heart Association’s scientific statement on SDoH in patients with heart failure emphasizes the need to better measure SDoH through the integration of data capture within clinical practice [22]. It also emphasizes the importance of clinician education on the impact of SDoH on health outcomes to motivate the capture of these data during clinical care. However, additional strategies will likely be needed to obtain these data given the cognitive overload currently experienced by clinicians [23,24]. Natural language processing interventions that reduce the burden on clinicians and enhance patient engagement are being explored as a promising means of automating SDoH data capture [25]. More sophisticated and granular geospatial indices in addition to the overall social deprivation index are also becoming available [26]. In addition, recent changes to Medicare reimbursement, including payment for Patient Navigation and Community Health Improvement services, should also support expanded collection of SDoH data in the Medicare population [27].

Multiple studies have documented decreased access to virtual health interventions, or how the digital divide often disproportionally impacts racial and ethnic minorities and those of lower socioeconomic status [4]. This trend has been found in the heart failure population as well, although it is not as robustly documented [28,29]. However, this aspect of some telehealth interventions may be changing. A recently published analysis of the CONNECT-HF mHealth study found that younger, non-White, urban participants and those with either Medicaid or no insurance had higher mHealth access. This may be in part due to the technology used being cell phones which have been widely available for some time [28]. This is to some extent aligned with our own findings that virtual care was correlated with the use of cell phones. However, there are multiple possible explanations for this finding, and it should be interpreted with caution. It may be specific to our health system, the management of the disease of HFrEF, or reflect the dissemination of digital technology in society over time, among other possible explanations. Future studies should work to understand the replicability of this finding and identify the reasons for the results observed so the learning can be applied to other contexts in which the disparities still exist and inform interventions to resolve them.

The study benefits from a large sample size, including academic, rural, suburban, and community settings. However, findings may not be fully generalizable to other settings that have different care delivery models, access to specialists, or payer mix. For instance, the majority of patients in this cohort had commercial insurance or Medicare, but this may be a reflection of these patients being older, which is generalizable for the HFrEF population. It is unclear to what extent our findings, such as high rates of ambulatory specialty care over primary care, would be found in a health system with a higher rate of patients without insurance or with Medicaid. As is the case with many US-based health systems, our findings are also limited to visits within the UCHealth health system and do not capture care received outside of the health system. Future studies should consider using alternate data sources such as claims data or health information exchanges to capture care received across more than one health system. Additionally, we used EF alone and not diagnosis codes to define our cohort and thus may have included some patients with very transient decreases in EF due to other reasons (eg myocarditis). However, this likely only represents a small number of patients.

Missing SDoH data is both an important finding and a limitation of this study. It has been mitigated as a limitation by the use of the SVI within our analysis. However, our inability to accurately measure these patient characteristics restricts our ability to fully evaluate for equity in care delivery. Notably, this data set’s limitations impact all evaluations of care within this large health system and likely reflect the current inadequacies of SDoH data capture in many other health systems [30]. Additionally, the low number of specific racial and ethnic minorities within the study cohort that participated in virtual visits limits our ability to detect differences in virtual visit use in these patient groups.

In conclusion, patients with HFrEF were more likely to receive care in person and with cardiology than virtually or with primary care. While patients with higher levels of digital access were more likely to be seen virtually, indicators of the digital divide did not correlate with race, ethnicity, or insurance status. Future studies should explore the replicability of these findings.