From November to December 2020, we conducted a retrospective cohort study of unique patient visits among 8 attending telemedicine champions (to ensure adequate distribution of modalities) at 3 primary care practices in an academic medical center in New York City serving a predominantly (>80%) Medicaid population. We randomly selected 2-3 mixed-modality clinical sessions (ie, scheduled in-person, video, and phone visits) per provider. Included visits were follow-ups with established patients of attending primary care providers (PCPs); modality was determined by PCP recommendation or patient preference. We excluded other visit types (eg, same-day sick visits, new patient encounters, resident or trainee visits, and visits with nonestablished patients) to ensure that all visits were conducted by the patient’s PCP, allotted the same duration (~20 min), and used the same template in the electronic health record across modalities (Figure 1).

Outcomes were informed by the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework [16]. We assessed reach (demographic differences by modality) and adoption (telemedicine uptake at the clinic, provider, and patient levels). For effectiveness, we identified United States Preventive Services Task Force and Accountable Care Organization quality metrics (ie, screening for blood pressure [BP], with BP<140/90 mmHg; depression; hemoglobin A_1c [HbA_1c]; lipids; mammography; cervical cancer; and colorectal cancer). We then conducted retrospective chart reviews to assess at-visit BP and depression screening as well as HbA_1c, lipid, and cancer screening orders. For implementation, we described conversion of video to phone visits and no-show rates. Given the ongoing pandemic, we obtained provider feedback on barriers and facilitators using rapid-cycle qualitative research methods to reduce time commitment and intensity of data collection required of providers and to inform real-time practice changes [17,18]. Following clinical sessions, providers were emailed a web-based form to input free-text narrative comments on barriers to and facilitators for each encounter; the form was returned 1-7 days after the visit (Multimedia Appendix 1). For maintenance, we examined adoption rates in a random sample of visits identified using the same inclusion/exclusion criteria over an equivalent period, 2 years later (November-December 2022).

Ethics approval was obtained from the Columbia University Irving Medical Center Institutional Review Board (IRB-AAAR5570). A waiver for informed consent was granted given the retrospective nature of our study. All data were properly secured and stored in a manner compliant with HIPAA (Health Insurance Portability and Accountability Act) on an encrypted server.

Overall, 24.6% (69/281) of the visits were in person, 25.3% (71/281) were through video, and 32.7% (92/281) through phone calls; 17.4% (49/281) were no-shows; 27.8% (78/281) of patients had no prior telemedicine visits. Patients aged ≥65 years (vs those aged <65 years) were more likely to have in-person (46/69, 67% vs 23/69, 33%) and phone visits (52/92, 57% vs 40/92, 43%) and less likely to have video (26/71, 37% vs 45/71, 63%) and no-show visits (22/49, 45% vs 27/49, 55%; P=.002). Non-English–speaking patients (vs English-speaking patients) were marginally more likely to have in-person (45/69, 65% vs 24/69, 35%), phone (63/92, 68% vs 29/92, 32%), and no-show visits (31/49, 63% vs 18/49, 37%); the proportion of video visits was almost similar between the two groups (35/71, 49% vs 36/71, 51%; P=.08). PCP was significantly associated with modality (P<.001). Clinic (P=.65), race (P=.39), and ethnicity (P=.81) were not significantly associated with modality.

We found marginally significant differences in BP screening for video (15/71, 21%), phone (16/92, 17%), and in-person (39/69, 57%) visits (adjusted P=.06), with significantly lower odds for video (adjusted odds ratio [AOR] 0.21, 95% CI 0.09-0.50; P<.001) and phone (AOR 0.16, 95% CI 0.08-0.30; P<.001) versus in-person visits. Similar trends emerged for depression screening (24/71, 34%; 12/92, 13%; and 41/69, 59%, respectively; adjusted P=.06), for video (AOR 0.35, 95% CI 0.14-0.86; P=.02) and phone (AOR 0.10, 95% CI 0.04-0.26; P<.001) versus in-person visits. For HbA_1c or lipid screening orders, we found significantly lower odds for phone versus in-person visits only (AOR 0.46, 95% CI 0.24-0.88; P=.02). We found no significant difference by modality for a combined metric of any cancer screening order (ie, mammography, colorectal cancer, or cervical cancer; Table 1).

Overall, 39.4% (56/142) of scheduled video visits converted to phone visits. We found no significant difference in no-show rates by modality (P=.17). Emerging themes included technological issues, patient preferences, and need for social support. Interpreters or relatives serving as translators were often identified as facilitators for non-English–speaking patients versus English-speaking patients (Table 2).

After 2 years, the distribution of in-person, video, and phone visits was 90.2% (111/123), 7.3% (9/123), and 2.4% (3/123), respectively.

After the first COVID-19 wave, we found similar distributions of in-person, video, and phone visits in low SES settings, with marginally better BP and depression screening for in-person care. De-adoption was marked within 2 years in our population.

Research delineating audio-only telemedicine and quality metrics is limited [5-7]. Our mixed methods findings highlight competing, disruptive technological barriers (ie, over one-third of video visits converted to audio-only visits) in low SES settings, potentially impacting quality metrics. We particularly observed gaps in screenings supported by ancillary clinic staff during in-person rooming (eg, medical assistants routinely perform BP and depression screenings), which was unavailable for telemedicine in our setting. Prior research shows an association between medical assistant–supported virtual rooming and successful video visit connections [23]; future studies should assess impact of virtual rooming on quality outcomes. Modality may be a proxy for other provider (eg, cognitive load, engagement, or shared decision-making) and patient factors (eg, transportation, access, or literacy), further impacting lab and cancer screening orders, which providers knew warranted travel. Some providers were more likely to use telemedicine, perhaps due to differences in preference, experience, or technologically savvy patient panels.

Our qualitative findings support prior qualitative research on determinants of telemedicine and remote patient monitoring, expanding on perspectives in historically marginalized populations [3,24,25]. Experts should examine whether addressing barriers (eg, technology access), promoting facilitators (eg, social support), and fine-tuning hybrid in-person and telemedicine models improve quality metrics and long-term telemedicine uptake. Future studies should assess within-modality differences in quality metrics by patient race, ethnicity, and language, given the documented disparities in telemedicine modality uptake. Special consideration is warranted for low-income settings, where in-person care equalizers may be unavailable (eg, video translator services, at-home labs, or device loan programs).

Limitations include unmeasured differences between modalities, including visit quality (eg, nonverbal communication and patient-provider rapport) and duration. Our sample size likely impacted cancer screening results, as few patients were due for screening. We were unable to assess provider or patient modality preferences (eg, providers less comfortable with video-based telemedicine scheduling more phone visits, older patients preferring in-person care, and technologically savvy patients using video visits), though we adjusted for clustering by provider, patient demographics, and prior telemedicine use. Additionally, telemedicine champions were used to ensure adequate distribution of modalities, which introduces bias; gaps are likely wider, overall. Lastly, we only describe 2 snapshots in time, albeit our results suggest telemedicine de-adoption in our population. Modality preferences, technological barriers, and restructuring of clinic workflows following the pandemic may have contributed to the reversion to in-person care and should be assessed as potential drivers of de-adoption.

Proponents of widespread telemedicine implementation should consider telemedicine impact on quality outcomes, provider and patient preferences, and mitigating strategies in historically marginalized populations.