Background: Funding changes in response to the COVID-19 pandemic supported the growth of direct-to-consumer virtual walk-in clinics in several countries. Little is known about patients who attend virtual walk-in clinics or how these clinics contribute to care continuity and subsequent health care use.
Objective: The objective of the present study was to describe the characteristics and measure the health care use of patients who attended virtual walk-in clinics compared to the general population and a subset that received any virtual family physician visit.
Methods: This was a retrospective, cross-sectional study in Ontario, Canada. Patients who had received a family physician visit at 1 of 13 selected virtual walk-in clinics from April 1 to December 31, 2020, were compared to Ontario residents who had any virtual family physician visit. The main outcome was postvisit health care use.
Results: Virtual walk-in patients (n=132,168) had fewer comorbidities and lower previous health care use than Ontarians with any virtual family physician visit. Virtual walk-in patients were also less likely to have a subsequent in-person visit with the same physician (309/132,168, 0.2% vs 704,759/6,412,304, 11%; standardized mean difference [SMD] 0.48), more likely to have a subsequent virtual visit (40,030/132,168, 30.3% vs 1,403,778/6,412,304, 21.9%; SMD 0.19), and twice as likely to have an emergency department visit within 30 days (11,003/132,168, 8.3% vs 262,509/6,412,304, 4.1%; SMD 0.18), an effect that persisted after adjustment and across urban/rural resident groups.
Conclusions: Compared to Ontarians attending any family physician virtual visit, virtual walk-in patients were less likely to have a subsequent in-person physician visit and were more likely to visit the emergency department. These findings will inform policy makers aiming to ensure the integration of virtual visits with longitudinal primary care.
Virtual walk-in clinics provide direct-to-consumer video, phone, or text-based physician consultations, often through a mobile phone app, and typically do not have a physical location. Prior to COVID-19, virtual walk-in clinics ostensibly helped meet a primary care need for people without a family physician or those who could not access their physician in a timely way, including those in rural settings [- ]. In Canada, Australia, and the United States, new COVID-19–related physician billing codes, intended to support virtual visits within existing primary care relationships, also drove a proliferation of virtual walk-in clinics [ , - ]. Many patients like virtual visits, particularly with their own physician, as they do not have to take time off work, arrange childcare, travel long distances, or pay for parking [ - ].
Despite these positive perceptions, there remain concerns about the quality of care provided through virtual visits in general, and in particular the care provided by large, corporate virtual walk-in clinics [, ]. These clinics offer an exclusively virtual experience, typically outside of existing primary care relationships, with no option for having an in-person exam [ ]. The lack of a physical exam has raised questions as to whether and how virtual encounters meet the standard of care for higher-acuity presentations [ ]. Exclusively virtual walk-in clinics typically do not integrate with patients’ existing sources of primary care, raising concerns about duplication and potential harm resulting from care discontinuity [ ]. Virtual visits may also exacerbate inequities in access resulting from language discordance, technological access, or literacy level [ , - ]. Additionally, the 24/7 access afforded by virtual walk-in clinics may prompt visits for transient, low-acuity medical symptoms that previously would not have occurred at all [ ], raising total system costs—a phenomenon known as “supplier-induced demand” [ ].
Although other studies have described the rapid expansion of virtual care, previous reports could not distinguish corporate virtual walk-in clinic visits from other virtual primary-care visits, including those with a patient’s own physician [, ]. Little is known about the physicians and patients who use exclusively virtual walk-in clinics. Our objectives were twofold: (1) describe the family physicians working in virtual walk-in clinics and compare them to the broader family physician pool and (2) describe the characteristics and health care use of patients using virtual walk-in clinics compared to the general population and a subset that received any virtual family physician visit.
Study Design and Setting
We conducted a retrospective, cross-sectional study of all Ontario residents and those who had encounters at any of 13 selected virtual walk-in clinics.
Ontario is Canada’s most populous province, with over 14.5 million residents. Provincial health insurance is provided without premiums or copayments to all citizens and permanent residents and covers emergency department visits, hospitalizations, and all medically necessary physician care. Most primary care is provided by family physicians, and nearly 80% of the population is enrolled to a family physician working in a patient enrollment model .
Prior to April 2020, use of an approved platform (the Ontario Telemedicine Network ) and a video (rather than phone) visit were requirements to bill for a virtual visit. After the onset of COVID-19, the Ontario Ministry of Health introduced several new temporary physician billing codes for synchronous virtual visits by video or phone with a value equivalent to that of in-person visits ( [ - , ], Tables S1A-B). Since then, the majority of publicly funded virtual visits have been conducted by phone [ , ]. Asynchronous visits (ie, provided by email or text message) are not covered by provincial insurance.
To recruit patient partners, we advertised through ICES (formerly known as the Institute for Clinical Evaluative Sciences) and selected 4 individuals with diverse backgrounds in gender, race, location, profession, and lesbian, gay, bisexual, transgender, queer/questioning, and other sexual identity (LGBTQ+) status. They also all had previous experience as patients at walk-in clinics. The patient partners, through meetings and email correspondence with the principal investigator, reviewed the analytic plan and contributed to results interpretation.
This study was approved by the Women’s College Hospital Research Ethics Board (REB 2020-0095-E).
Population-based health administrative data sets were linked using unique encoded identifiers and analyzed at ICES in Ontario, Canada (, Table S2 lists the databases).
We developed a noncomprehensive list of virtual walk-in clinics by searching business names obtained from a list of group billing numbers and corresponding group names provided by the Ontario Ministry of Health. We used this list to identify all groups with “virtual” or “tele” in their name, used Google to search for the identified names, and reviewed the clinic websites to determine which provided exclusively virtual care (ie, without the possibility of an in-person office visit with a physician). In addition, we used Google to search for the combined terms “Canada” or “Ontario” AND “virtual clinic” or “telemedicine,” identifying several other groups for inclusion for a total of 20 virtual-only walk-in clinics. We then restricted the list to groups that had active billing claims during the period from April 1, 2019, to December 31, 2020 (n=13).
We included all family physicians with at least 5 virtual billing claims under one of our included virtual walk-in clinics from April 1 to December 31, 2020. The comparison group was all family physicians with active billings during this time.
Virtual Walk-in Clinic Patients
We selected all patients who received at least one family physician visit at 1 of the 13 included virtual walk-in clinics from April 1 to December 31, 2020. The comparison group was all Ontario residents with an active health card and a health care contact within the previous 8 years as of April 1, 2020. For measures of health care use, we restricted the Ontario population to those who had at least one virtual family physician visit from April 1 to December 31, 2020.
Patient Characteristics and Health Care Use
We report the following patient characteristics: age, sex, neighborhood income quintile, urban or rural residence , and whether they were a recent provincial insurance registrant (within the past 10 years), a proxy measure for recent immigration [ ]. We also examined the count of comorbidities using Johns Hopkins aggregated diagnosis groups (obtained from the Johns Hopkins ACG System, version 10) and prior health care use using adjusted clinical group (ACG) resource utilization bands (RUBs) over the previous 2 years [ ]. We describe patient enrollment status, enrollment model type, and continuity of care using the Usual Provider Continuity metric [ ] ( , Table S3 shows operational definitions of all variables).
For patients with more than one virtual walk-in clinic visit, we randomly selected one virtual walk-in clinic visit and excluded all others (, Figure S1). For the Ontario population comparison, characteristics were anchored to April 1, 2020, and for characteristics that required anchoring to an encounter, we randomly selected one family physician virtual visit and excluded all others.
We report the frequencies of the top 10 most common medical diagnoses in each group. We also report whether virtual encounters were with a patient’s enrolling family physician, the encounter day of the week, and 30-day postvisit health care use, including repeat virtual visits, office visits, low-acuity emergency department visits (defined as a Canadian Triage and Acuity Scale score of 4 to 5 ), any emergency department visit, or urgent hospitalization.
We compared the characteristics of physicians who provided a virtual walk-in clinic visit to all family physicians with active billings. We also compared virtual walk-in clinic patients to the general Ontario population and the subset of the population that received any virtual family physician visit. Finally, we stratified health care use variables by the patients’ urban/rural residence status (large urban, small urban, or rural), because this is known to be associated with rates of emergency department use .
To compare groups, we used standardized mean differences (SMDs) and considered differences greater than 10% (0.1) to be significant . SMDs have the advantage of quantifying the magnitude of differences between groups—this is particularly useful in studies with large sample sizes, where even very small differences can result in a P value <.05. To examine the adjusted association between type of virtual visit and emergency department use in the subsequent 30 days, we used logistic regression with generalized estimating equations (GEEs) to account for clustering by the index virtual visit physician. Any patient who received both types of virtual visits was removed from the “other virtual” visit group, such that each individual appeared only once. We stratified the regression by large urban, small urban, or rural residence and adjusted for patient age, sex, neighborhood income quintile, RUB, and recent provincial insurance registrant status. Observations with missing income quintile (0.2%) were not included in the regression.
Analyses were executed in SAS (version 9.4; SAS Institute Inc).
Virtual Walk-in Clinic Volumes Over Time
From April 2019 to December 2020, the weekly volume of patients increased 2-fold (). The number of individual physicians providing virtual encounters at virtual walk-in clinics each week increased sharply between March and May 2020, and by November 2020 was 2.5 times higher than in February 2020.
Physicians Working in Virtual Walk-in Clinics
Compared to the overall Ontario population of family physicians with active billing between April 1 and December 31, 2020 (N=14,825;), virtual walk-in clinic physicians were younger, more likely to have graduated within the past 10 years, and more likely to practice in a large urban setting. They were also considerably more likely to work fee-for-service, rather than as part of a patient enrollment model. There was no significant difference in the number of patients seen per day.
|Physician characteristics||Provided >5 virtual walk-in clinic visits (n=242)||All family physicians with active billing (N=14,825)||Standardized mean differencea|
|Physician age (years)|
|Mean (SD)||40.3 (10.9)||49.3 (14.0)||0.72|
|Median (IQR)||37 (32-46)||48 (38-59)||0.75|
|Physician age group (years), n (%)|
|25-34||102 (42.1)||2471 (16.7)||0.58|
|35-49||95 (39.3)||5285 (35.6)||0.07|
|50-64||34 (14)||4843 (32.7)||0.45|
|≥65||11 (4.5)||2226 (15)||0.36|
|Physicians self-reporting female gender, n (%)||119 (49.2)||7112 (48)||0.02|
|Time since physician graduated medical school (years), n (%)|
|0-5||37 (15.3)||764 (5.2)||0.34|
|6-10||57 (23.6)||2280 (15.4)||0.21|
|11-20||32 (13.2)||2939 (19.8)||0.18|
|21-30||35 (14.5)||3027 (20.4)||0.16|
|≥31||20 (8.3)||4446 (30)||0.57|
|Missing||61 (25.2)||1369 (9.2)||0.43|
|Physician practice location, n (%)|
|Large urban||202 (83.5)||11,010 (74.3)||0.23|
|Small urban||25 (10.3)||2303 (15.5)||0.16|
|Rural||9 (3.7)||1043 (7)||0.15|
|Missing||6 (2.5)||469 (3.2)||0.04|
|Physician primary care model, n (%)|
|Enhanced fee-for-service||58 (24)||2753 (18.6)||0.13|
|Capitation||40 (16.5)||4665 (31.5)||0.36|
|Team-based||0 (0)||1824 (12.3)||0.53|
|Fee-for-service (no enrollment)||136 (56.2)||4926 (33.2)||0.47|
|Other||8 (3.3)||657 (4.4)||0.06|
|Number of patients seen per day as virtual visits, median (IQR)||12 (5-22)||13 (7-22)||0.05|
aA standardized mean difference of at least 10% (0.1) was considered to indicate a significant difference.
Patients Attending Virtual Walk-in Clinics
Compared to the overall Ontario population, patients who attended a virtual walk-in clinic visit were more likely to be young adults and less likely to be children or older adults (). Virtual walk-in clinic patients were also more likely to be female and live in a small urban setting. The proportion of virtual walk-in clinic patients that were new registrants or that resided in low-income neighborhoods did not differ from the overall Ontario population.
Virtual walk-in clinic patients were less likely to be enrolled to a family physician than the Ontario population (84,861/132,168, 64.2% vs 10,908,871/14,709,408, 74.2%; SMD 0.22) and had lower continuity of care (SMD 0.44). Less than 0.1% (64/132,168) of virtual walk-in visits were with the patient’s enrolling family physician.
Compared to all Ontarians who had any virtual family physician visit, virtual walk-in clinic patients had fewer comorbidities (73,526/132,168, 55.6% vs 3,207,972/6,412,304, 50% were “low”; SMD 0.11) and lower levels of previous health care use (38,584/132,168, 29.2% vs 1,358,312/6,412,304, 21.2% were “low”; SMD 0.19). They were also more likely to have their virtual visit on a Saturday or Sunday.
|Characteristics||Visited a virtual walk-in clinic (n=132,168)||Ontario population overall (N=14,709,408)||Standardized mean difference a|
|Age (years), mean (SD)||38.8 (19.4)||41.3 (23.0)||0.12|
|Age group (years), n (%)|
|<18||13,730 (10.4)||2,761,674 (18.8)||0.24|
|18-29||35,300 (26.7)||2,217,008 (15.1)||0.29|
|30-44||35,980 (27.2)||3,020,751 (20.5)||0.16|
|45-64||31,714 (24)||4,065,422 (27.6)||0.08|
|65-74||9744 (7.4)||1,497,270 (10.2)||0.10|
|≥75||5700 (4.3)||1,147,283 (7.8)||0.15|
|Female sex, n (%)||79,246 (60)||7,472,638 (50.8)||0.18|
|Neighborhood income quintile, n (%)|
|Lowest (1)||29,822 (22.6)||2,890,652 (19.7)||0.07|
|2||26,598 (20.1)||2,887,125 (19.6)||0.01|
|3||25,667 (19.4)||2,966,912 (20.2)||0.02|
|4||26,141 (19.8)||2,970,860 (20.2)||0.01|
|Highest (5)||23,669 (17.9)||2,968,321 (20.2)||0.06|
|Missing||271 (0.2)||25,538 (0.2)||0.01|
|Recent provincial insurance registrant (past 10 years), n (%)||14,334 (10.8)||1,352,790 (9.2)||0.05|
|Residence, n (%)|
|Large urban||83,484 (63.2)||10,758,196 (73.1)||0.22|
|Small urban||37,581 (28.4)||2,781,005 (18.9)||0.23|
|Rural||9863 (7.5)||1,025,197 (7)||0.02|
|Missing||1240 (0.9)||145,010 (1)||0|
|Comorbidity count, n (%)b|
|Low (0-5)||73,526 (55.6)||3,207,972 (50)||0.11|
|Moderate (6-9)||39,883 (30.2)||2,203,659 (34.4)||0.09|
|High (≥10)||18,759 (14.2)||1,000,673 (15.6)||0.04|
|Health care utilization band, n (%)b|
|Low (0-2)||38,584 (29.2)||1,358,312 (21.2)||0.19|
|Moderate (3)||63,896 (48.3)||3,389,803 (52.9)||0.09|
|High (4-5)||29,688 (22.5)||1,664,189 (26)||0.08|
|Enrolled to a family physician, n (%)||84,861 (64.2)||10,908,871 (74.2)||0.22|
|Primary care enrollment model type, n (%)|
|Capitation||35,159 (26.6)||4,241,999 (28.8)||0.05|
|Enhanced fee-for-service||43,114 (32.6)||4,412,144 (30.0)||0.06|
|Team-based||27,508 (20.8)||3,588,850 (24.4)||0.09|
|Other group||499 (0.4)||99,775 (0.7)||0.04|
|Fee-for-service (no enrollment)||18,083 (13.7)||1,038,591 (7.1)||0.22|
|No prior physician primary care||7805 (5.9)||1,328,049 (9)||0.12|
|Continuity of care|
|Continuity (%), median (IQR)||50 (30-83.3)||75 (50-100)||0.44|
|Missing (<2 visits), n (%)||25,119 (19)||4,160,139 (28.3)||0.22|
|Day of week of visit, n (%)b|
|Monday||22,758 (17.2)||1,291,840 (20.1)||0.08|
|Tuesday||23,089 (17.5)||1,386,411 (21.6)||0.10|
|Wednesday||21,799 (16.5)||1,195,007 (18.6)||0.06|
|Thursday||21,396 (16.2)||1,292,295 (20.2)||0.10|
|Friday||20,539 (15.5)||887,193 (13.8)||0.05|
|Saturday||12,227 (9.3)||253,311 (4)||0.21|
|Sunday||10,360 (7.8)||106,247 (1.7)||0.29|
|Index virtual visit was with enrolling physician, n (%)b||64 (0)||3,949,998 (61.6)||1.79|
aA standardized mean difference of at least 10% (0.1) was considered to indicate a significant difference.
bFor all variables related to the index visit (including comorbidity count and health care utilization band), the Ontario population group was restricted to those with any virtual family physician visit from April 1 to December 31, 2020 (n=6,412,304).
Top 10 Diagnoses at Virtual Encounters
Diagnoses at virtual walk-in clinic visits were similar to those for all Ontarians’ virtual family physician visits (). However, acute conditions occurred more commonly among virtual walk-in clinic visits, and chronic disease diagnoses were more common among all virtual family physician visits.
|Diagnoses||Values, n (%)|
|Virtual walk-in visit (n=132,168)|
|Other ill-defined conditions||13,837 (10.5)|
|Mental health||5226 (4)|
|Acute nasopharyngitis, common cold||4838 (3.7)|
|Gastrointestinal symptomsa||3598 (2.7)|
|Other disorders of the urinary tract||3518 (2.7)|
|Essential, benign hypertension||3415 (2.6)|
|Cellulitis, abscess||2993 (2.3)|
|Family planning, contraceptive advice, advice on sterilization, abortion||2778 (2.1)|
|Ontario population with virtual family physician visit in 2020 (n=6,412,304)|
|Mental healthb||488,468 (7.6)|
|Other ill-defined conditions||393,541 (6.1)|
|Essential, benign hypertension||372,793 (5.8)|
|Diabetes mellitus, including complications||330,292 (5.2)|
|Musculoskeletal symptoms other than back painc||225,615 (3.5)|
|Gastrointestinal symptomsa||202,921 (3.2)|
|Eczema, atopic dermatitis, neurodermatitis||140,262 (2.2)|
|Disorders of lipid metabolism||136,877 (2.1)|
|Acute nasopharyngitis, common cold||135,912 (2.1)|
|Lumbar strain, lumbago, coccydynia, sciatica||107,305 (1.7)|
aGastrointestinal symptoms included anorexia, nausea and vomiting, heartburn, dysphagia, hiccup, hematemesis, jaundice, ascites, abdominal pain, melena, and masses.
bMental health included anxiety, neurosis, hysteria, neurasthenia, obsessive compulsive neurosis, and reactive depression.
cMusculoskeletal symptoms other than back pain included leg cramps, leg pain, muscle pain, joint pain, arthralgia, joint swelling, and masses.
Health Care Use Following First Virtual Visit
Patients of virtual walk-in clinics had more repeat virtual visits within 30 days than Ontarians with a virtual family physician visit (40,030/132,168, 30.3% vs 1,403,304/6,412,304, 21.9%; SMD 0.19;). They were also considerably less likely to have an in-person visit with the same physician (309/132,168, 0.2% vs 704,759/6,412,304, 11%; SMD 0.48), with any physician (15,441/132,168, 11.7% vs 980,556/6,412,304, 15.3%; SMD 0.11), or with their own physician (5,633/132,168, 4.3% vs 584,993/6,412,304, 9.1%; SMD 0.20). Patients of virtual walk-in clinics were twice as likely to have any emergency department visit (11,003/132,168, 8.3% vs 262,509/6,412,304, 4.1%; SMD 0.18), with similar results across urban/rural residence strata ( , Table S4). Virtual walk-in patients were also twice as likely to have a low-acuity emergency department visit (3,517/132,168, 2.7% vs 69,425/6,412,304, 1.1%; SMD 0.12).
After adjustment, those who received a virtual walk-in clinic visit remained more likely to have an emergency department visit within 30 days in all 3 urban/rural residence strata: large urban (adjusted odds ratio [aOR] 2.26, 95% CI 2.08-2.45), small urban (aOR 2.08, 95% CI 1.99-2.18), and rural locations (aOR 1.87, 95% CI 1.69-2.07).
|Measures of use within 30 days following the virtual visit||Visit to a virtual walk-in clinic|
(n=132,168), n (%)
|Ontario population with virtual family physician visit in 2020 (n=6,412,304), n (%)||Standardized mean differencea|
|At least one repeated virtual visit with any physician||40,030 (30.3)||1,403,778 (21.9)||0.19|
|At least one in-person visit with same physician||309 (0.2)||704,759 (11)||0.48|
|At least one in-person visit with own enrolling physician||5633 (4.3)||584,993 (9.1)||0.20|
|At least one in-person visit with any physician||15,441 (11.7)||980,556 (15.3)||0.11|
|At least one emergency department visit||11,003 (8.3)||262,509 (4.1)||0.18|
|At least one low-acuity emergency department visit||3517 (2.7)||69,425 (1.1)||0.12|
|At least one urgent hospitalization||1178 (0.9)||49,717 (0.8)||0.01|
aA standardized difference of at least 10% (0.1) was considered to indicate a significant difference.
We compared patient characteristics and outcomes from visits to 13 virtual walk-in clinics to all virtual family physician visits in the Ontario population. Virtual walk-in patients were younger, were more likely to be female, and had lower continuity of care than the general population; they also had lower previous health care use than Ontario residents with any virtual family physician visit. Compared to Ontarians attending any family physician virtual visit, virtual walk-in patients were more likely to have a repeat virtual visit and less likely to have an in-person visit in the subsequent 30 days. They were also significantly more likely to visit the emergency department, a finding that held true in big cities, small towns, and rural areas, even after adjustment for potential confounders.
Our findings highlight two areas of potential concern with virtual walk-in clinics. The first is the lack of continuity of patient/physician relationships, a limitation shared with regular walk-in clinics. This is almost certainly accompanied by a lack of informational continuity, as presently there are no incentives or even regulatory frameworks compelling a virtual (or nonvirtual) walk-in physician to share information with a patient’s usual provider. Easy access to a family physician outside existing primary care relationships should be weighed against the risks of low-continuity care [- ]. Low-continuity care does not offer opportunities for longitudinal preventive care and has been associated with more adverse events among patients with chronic conditions like diabetes [ ].
The second major concern is the potential downstream consequences of a care model that operates without the possibility of a physical examination. Patients who have a virtual visit with their own family physician have more options for in-person follow-up. In the absence of a physical examination, physicians at virtual walk-in clinics may recommend that patients go to emergency departments to be examined. Alternatively, our finding of higher rates of emergency department visits among virtual walk-in clinic users could reflect the downstream consequences of an incorrect or delayed diagnosis. The absence of a physical examination also has the potential to negatively affect other dimensions of care quality  and lead to more inappropriate prescriptions [ - ], testing, follow-up visits [ ], and referrals to consultants. Supplier-induced demand through attractive marketing campaigns, combined with increased downstream health care use, could increase overall health care costs.
Reports from the United States, United Kingdom, and Sweden have described virtual-visit users as more likely to be healthy young adults [, ] with higher socioeconomic status [ , , ]. Although we similarly found that users were more likely to be young adults with lower levels of health care use, our findings do not suggest that publicly funded virtual walk-in visits are disproportionately serving the affluent.
Like others [, ], we found that virtual walk-in doctors were younger, with fewer years in practice than the average family physician. They were also more likely to be fee-for-service physicians, who would have experienced a sharp drop in income early in the pandemic due to decreased in-person visit volumes [ ]. Without the income stability offered by capitation payments, fee-for-service physicians likely turned to other revenue sources, including virtual walk-in clinics.
Our 4 patient partners provided several reasons why virtual walk-in clinics may be attractive to patients. They indicated that virtual walk-in clinics are convenient and require no travel, do not require scheduling an appointment or going through a “gatekeeper” to care such as an office assistant, might be more efficient when a patient is seeking a prescription or mental health care, and also provide relative anonymity to patients seeking care if they see a different physician on each visit. For these and other reasons, patients appreciate having the choice to visit a virtual walk-in clinic.
Developing a policy landscape that favors an efficient use of virtual visits is an urgent priority for health insurers [, , ]. In 2021, the Ontario Ministry of Health added virtual-visit codes to the “outside use” list, which financially penalizes capitation model physicians each time their patients see other family physicians [ ]. Policy makers could also consider significantly reducing the value of virtual-visit codes when used by physicians without a physical office location or without a preexisting primary care roster. Another option is for physician regulatory bodies to mandate that physicians offering virtual visits also offer in-person appointments, as was recently done in Manitoba [ ].
Our study has several limitations. First, there are likely many more physicians and patients participating in virtual walk-in clinic care; however, because they are either not linked to a group billing number or are privately paid , we had no way of identifying them for study inclusion. Second, we could not distinguish video from phone visits, as these were claimed using the same billing code. We further could not capture text or email consultations, as these are ineligible for coverage by provincial insurance. Third, we exclusively focused on family physicians, as these are the most common providers of primary care in Ontario, and did not assess visits to pediatricians or psychiatrists. Finally, our findings are most generalizable to other settings with publicly funded virtual walk-in visits.
The number of Ontario patients and family physicians participating in a sample of virtual walk-in clinics rose rapidly after COVID-19–related physician fee schedule changes. Our findings suggest that these visits were associated with increased emergency department use. To ensure virtual walk-in clinics contribute positively to health outcomes and health system efficiency, policy makers should prioritize regulations and billing changes that ensure the integration of virtual and in-person visits while promoting continuity of care.
We would like to thank our patient partners CB, Jerome Johnson, Krysta Nesbitt, and Patrick Roncal for their contributions to the discussion about the study findings. We thank Alexander Kopp from ICES for his guidance in data analysis and methodology. We also thank our knowledge users Ed Jess, David Kaplan, David Price, Sundeep Banwatt, and Carolyn Canfield for their contributions to results interpretation. This project is supported by the Canadian Institutes of Health Research (project grant 175285). LLS is supported by the University of Toronto Department of Medicine, the Toronto General Hospital Research Institute, the Women’s College Institute for Health System Solutions and Virtual Care, and the Peter Gilgan Centre for Women’s Cancers at Women’s College Hospital, in partnership with the Canadian Cancer Society. TK is the Fidani Chair of Improvement and Innovation at the University of Toronto. She is supported as a clinician scientist by the Department of Family and Community Medicine at St. Michael’s Hospital and the University of Toronto. NMI is supported by a Canada Research Chair in Implementation of Evidence-based Practice and a Clinician Scholar award from the Department of Family and Community Medicine at Women’s College Hospital and the University of Toronto. This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health (MOH) and the Ministry of Long-Term Care. Parts of this material are based on data and information compiled and provided by the MOH, the Canadian Institute for Health Information, and Ontario Health. The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred.
The data set from this study is held securely in coded form at ICES. While legal data sharing agreements between ICES and data providers (eg, health care organizations and government) prohibit ICES from making the data set publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at www.ices.on.ca/DAS (email: email@example.com). The full data set creation plan and underlying analytic code are available from the authors upon request, understanding that the computer programs may rely upon coding templates or macros that are unique to ICES and are therefore either inaccessible or may require modification.
Conflicts of Interest
Supplementary materials.DOCX File , 40 KB
- Mehrotra A, Bhatia RS, Snoswell CL. Paying for telemedicine after the pandemic. JAMA 2021 Feb 02;325(5):431-432 [FREE Full text] [CrossRef] [Medline]
- Chu C, Cram P, Pang A, Stamenova V, Tadrous M, Bhatia RS. Rural telemedicine use before and during the COVID-19 pandemic: repeated cross-sectional study. J Med Internet Res 2021 Apr 05;23(4):e26960 [FREE Full text] [CrossRef] [Medline]
- Mehrotra A, Jena AB, Busch AB, Souza J, Uscher-Pines L, Landon BE. Utilization of telemedicine among rural medicare beneficiaries. JAMA 2016 May 10;315(18):2015-2016 [FREE Full text] [CrossRef] [Medline]
- Matthewman S, Spencer S, Lavergne MR, McCracken RK, Hedden L. An environmental scan of virtual "walk-in" clinics in canada: comparative study. J Med Internet Res 2021 Jun 11;23(6):e27259 [FREE Full text] [CrossRef] [Medline]
- Glazier RH, Green ME, Wu FC, Frymire E, Kopp A, Kiran T. Shifts in office and virtual primary care during the early COVID-19 pandemic in Ontario, Canada. CMAJ 2021 Feb 08;193(6):E200-E210 [FREE Full text] [CrossRef] [Medline]
- Patel SY, Mehrotra A, Huskamp HA, Uscher-Pines L, Ganguli I, Barnett ML. Trends in outpatient care delivery and telemedicine during the COVID-19 pandemic in the US. JAMA Intern Med 2021 Mar 01;181(3):388-391 [FREE Full text] [CrossRef] [Medline]
- Baum A, Kaboli PJ, Schwartz MD. Reduced in-person and increased telehealth outpatient visits during the COVID-19 pandemic. Ann Intern Med 2021 Jan;174(1):129-131 [FREE Full text] [CrossRef] [Medline]
- Wosik J, Fudim M, Cameron B, Gellad ZF, Cho A, Phinney D, et al. Telehealth transformation: COVID-19 and the rise of virtual care. J Am Med Inform Assoc 2020 Jun 01;27(6):957-962 [FREE Full text] [CrossRef] [Medline]
- Dullet NW, Geraghty EM, Kaufman T, Kissee JL, King J, Dharmar M, et al. Impact of a university-based outpatient telemedicine program on time savings, travel costs, and environmental pollutants. Value Health 2017 Apr;20(4):542-546 [FREE Full text] [CrossRef] [Medline]
- Reed ME, Huang J, Graetz I, Lee C, Muelly E, Kennedy C, et al. Patient characteristics associated with choosing a telemedicine visit vs office visit with the same primary care clinicians. JAMA Netw Open 2020 Jun 01;3(6):e205873 [FREE Full text] [CrossRef] [Medline]
- Kelley L, Phung M, Stamenova V, Fujioka J, Agarwal P, Onabajo N, et al. Exploring how virtual primary care visits affect patient burden of treatment. Int J Med Inform 2020 Sep;141:104228. [CrossRef] [Medline]
- McGrail KM, Ahuja MA, Leaver CA. Virtual visits and patient-centered care: results of a patient survey and observational study. J Med Internet Res 2017 May 26;19(5):e177 [FREE Full text] [CrossRef] [Medline]
- Evaluation of Babylon GP at Hand: final evaluation report. Ipsos MORI. 2019. URL: http://allcatsrgrey.org.uk/wp/wpfb-file/evaluation-of-babylon-gp-at-hand-final-report-pdf/ [accessed 2022-12-08]
- Rose S, Hurwitz HM, Mercer MB, Hizlan S, Gali K, Yu P, et al. Patient experience in virtual visits hinges on technology and the patient-clinician relationship: a large survey study with open-ended questions. J Med Internet Res 2021 Jun 21;23(6):e18488 [FREE Full text] [CrossRef] [Medline]
- Stamenova V, Agarwal P, Kelley L, Fujioka J, Nguyen M, Phung M, et al. Uptake and patient and provider communication modality preferences of virtual visits in primary care: a retrospective cohort study in Canada. BMJ Open 2020 Jul 06;10(7):e037064 [FREE Full text] [CrossRef] [Medline]
- Herzer KR, Pronovost PJ. Ensuring quality in the era of virtual care. JAMA 2021 Feb 02;325(5):429-430. [CrossRef] [Medline]
- Hardcastle L, Ogbogu U. Virtual care: Enhancing access or harming care? Healthc Manage Forum 2020 Nov 20;33(6):288-292 [FREE Full text] [CrossRef] [Medline]
- Standard of Practice: Virtual Medicine. The College of Physicians and Surgeons of Manitoba. URL: https://cpsm.mb.ca/assets/Standards%20of%20Practice/Standard%20of%20Practice%20Virtual%20Medicine.pdf [accessed 2022-12-08]
- Eberly LA, Kallan MJ, Julien HM, Haynes N, Khatana SAM, Nathan AS, et al. Patient characteristics associated with telemedicine access for primary and specialty ambulatory care during the COVID-19 pandemic. JAMA Netw Open 2020 Dec 01;3(12):e2031640 [FREE Full text] [CrossRef] [Medline]
- Kalicki AV, Moody KA, Franzosa E, Gliatto PM, Ornstein KA. Barriers to telehealth access among homebound older adults. J Am Geriatr Soc 2021 Sep;69(9):2404-2411 [FREE Full text] [CrossRef] [Medline]
- Park J, Erikson C, Han X, Iyer P. Are state telehealth policies associated with the use of telehealth services among underserved populations? Health Aff (Millwood) 2018 Dec;37(12):2060-2068. [CrossRef] [Medline]
- Dorn SD. Backslide or forward progress? Virtual care at U.S. healthcare systems beyond the COVID-19 pandemic. NPJ Digit Med 2021 Jan 08;4(1):6 [FREE Full text] [CrossRef] [Medline]
- Bhatia RS, Chu C, Pang A, Tadrous M, Stamenova V, Cram P. Virtual care use before and during the COVID-19 pandemic: a repeated cross-sectional study. CMAJ Open 2021;9(1):E107-E114 [FREE Full text] [CrossRef] [Medline]
- Kiran T, Kopp A, Glazier RH. Those left behind from voluntary medical home reforms in Ontario, Canada. Ann Fam Med 2016 Nov;14(6):517-525 [FREE Full text] [CrossRef] [Medline]
- Glazier RH, Kopp A, Schultz SE, Kiran T, Henry DA. All the right intentions but few of the desired results: lessons on access to primary care from Ontario’s patient enrolment models. Healthcare Q 2012;15:17-21.
- ICES. Corporate Provider Database (CPDB). ICES Data Dictionary. 2021. URL: https://datadictionary.ices.on.ca/Applications/DataDictionary/Library.aspx?Library=CPDB [accessed 2022-12-28]
- Juurlink DN, Preyra C, Croxford R, Chong A, Austin PC, Tu JV, et al. Canadian Institute for Health Information Discharge Abstract Database: A Validation Study. Institute for Clinical Evaluative Sciences. 2006. URL: "https://www.ices.on.ca/Pu blications/Atlases-and-Reports/2006/Canadian-Institute-for-Health-Information [accessed 2022-12-28] />
- ICES. Physician Database (IPDB). ICES Data Dictionary. 2018. URL: https://datadictionary.ices.on.ca/Applications/Data Dictionary/Library.aspx?Library=IPDB [accessed 2022-12-28] />
- CIHI Data Quality Study of Ontario Emergency Department Visits for Fiscal Year 2004–2005—Executive Summary. Canadian Institute for Health Information. 2007. URL: https://secure.cihi.ca/free_products/vol1_nacrs_executive_summary _nov2_2007.pdf [accessed 2022-12-28] />
- ICES. Ontario Health Insurance Plan (OHIP). ICES Data Dictionary. 2021. URL: https://datadictionary.ices.on.ca/Applica tions/DataDictionary/Library.aspx?Library=OHIP [accessed 2022-12-28] />
- ICES. Primary Care Population (PCPOP). ICES Data Dictionary. 2021. URL: https://datadictionary.ices.on.ca/Applications/DataDictionary/Library.aspx?Library=PCPOP [accessed 2022-12-28]
- Alter DA, Naylor CD, Austin P, Tu JV. Effects of Socioeconomic Status on Access to Invasive Cardiac Procedures and on Mortality after Acute Myocardial Infarction. New England Journal of Medicine 1999;341:1359-1367.
- Kralj B. Measuring Rurality - RIO2008_BASIC: Methodology and Results. OMA Economics Department. 2009. URL: https://policycommons.net/artifacts/1227502/measuring-rurality/1780574/ [accessed 2012-12-12]
- Agarwal P, Wang R, Meaney C, Walji S, Damji A, Gill N, et al. Sociodemographic differences in patient experience with primary care during COVID-19: results from a cross-sectional survey in Ontario, Canada. BMJ Open 2022 May 09;12(5):e056868 [FREE Full text] [CrossRef] [Medline]
- Kiran T, Wang R, Handford C. Keeping doors open: A cross-sectional survey of family physician practice patterns during COVID-19, needs, and intentions. medRxiv Preprint posted online on December 21, 2021. [CrossRef]
- Austin P, van Walraven C, Wodchis W, Newman A, Anderson G. Using the Johns Hopkins Aggregated Diagnosis Groups (ADGs) to predict mortality in a general adult population cohort in Ontario, Canada. Med Care 2011 Oct;49(10):932-939 [FREE Full text] [CrossRef] [Medline]
- Rodriguez HP, Marshall RE, Rogers WH, Safran DG. Primary care physician visit continuity: a comparison of patient-reported and administratively derived measures. J Gen Intern Med 2008 Sep;23(9):1499-1502 [FREE Full text] [CrossRef] [Medline]
- Steele S, Anstett D, Milne WK. Rural emergency department use by CTAS IV and V patients. CJEM 2008 May;10(3):209-214. [CrossRef] [Medline]
- Kiran T, Moineddin R, Kopp A, Frymire E, Glazier RH. Emergency department use and enrollment in a medical home providing after-hours care. Ann Fam Med 2018 Sep;16(5):419-427 [FREE Full text] [CrossRef] [Medline]
- Austin PC. Using the Standardized Difference to Compare the Prevalence of a Binary Variable Between Two Groups in Observational Research. Commun Stat Simul Comput 2009 Apr 09;38(6):1228-1234. [CrossRef]
- Pereira Gray DJ, Sidaway-Lee K, White E, Thorne A, Evans PH. Continuity of care with doctors-a matter of life and death? A systematic review of continuity of care and mortality. BMJ Open 2018 Jun 28;8(6):e021161 [FREE Full text] [CrossRef] [Medline]
- Nyweide DJ, Bynum JP. Relationship between continuity of ambulatory care and risk of emergency department episodes among older adults. Ann Emerg Med 2017 Apr;69(4):407-415.e3 [FREE Full text] [CrossRef] [Medline]
- Ionescu-Ittu R, McCusker J, Ciampi A, Vadeboncoeur A, Roberge D, Larouche D, et al. Continuity of primary care and emergency department utilization among elderly people. CMAJ 2007 Nov 20;177(11):1362-1368 [FREE Full text] [CrossRef] [Medline]
- Weir DL, McAlister FA, Majumdar SR, Eurich DT. The interplay between continuity of care, multimorbidity, and adverse events in patients with diabetes. Med Care 2016 Apr;54(4):386-393. [CrossRef] [Medline]
- Ray KN, Shi Z, Gidengil CA, Poon SJ, Uscher-Pines L, Mehrotra A. Antibiotic prescribing during pediatric direct-to-consumer telemedicine visits. Pediatrics 2019 May;143(5):e20182491 [FREE Full text] [CrossRef] [Medline]
- Martinez KA, Rood M, Jhangiani N, Kou L, Rose S, Boissy A, et al. Patterns of use and correlates of patient satisfaction with a large nationwide direct to consumer telemedicine service. J Gen Intern Med 2018 Oct;33(10):1768-1773 [FREE Full text] [CrossRef] [Medline]
- Uscher-Pines L, Mulcahy A, Cowling D, Hunter G, Burns R, Mehrotra A. Access and quality of care in direct-to-consumer telemedicine. Telemed J E Health 2016 Apr;22(4):282-287 [FREE Full text] [CrossRef] [Medline]
- Shi Z, Mehrotra A, Gidengil CA, Poon SJ, Uscher-Pines L, Ray KN. Quality of care for acute respiratory infections during direct-to-consumer telemedicine visits for adults. Health Aff (Millwood) 2018 Dec;37(12):2014-2023 [FREE Full text] [CrossRef] [Medline]
- Jain T, Mehrotra A. Comparison of direct-to-consumer telemedicine visits with primary care visits. JAMA Netw Open 2020 Dec 01;3(12):e2028392 [FREE Full text] [CrossRef] [Medline]
- Alexander GC, Tajanlangit M, Heyward J, Mansour O, Qato DM, Stafford RS. Use and content of primary care office-based vs telemedicine care visits during the COVID-19 pandemic in the US. JAMA Netw Open 2020 Oct 01;3(10):e2021476 [FREE Full text] [CrossRef] [Medline]
- Dahlgren C, Dackehag M, Wändell P, Rehnberg C. Determinants for use of direct-to-consumer telemedicine consultations in primary healthcare-a registry based total population study from Stockholm, Sweden. BMC Fam Pract 2021 Jun 26;22(1):133 [FREE Full text] [CrossRef] [Medline]
- Bhatia RS, Jamieson T, Shaw J, Piovesan C, Kelley L, Falk W. Canada's Virtual Care Revolution: A Framework for Success. CD Howe Institute. URL: https://www.cdhowe.org/sites/default/files/2021-12/Commentary_586.pdf [accessed 2022-12-08]
- Bhatia RS, Falk W, Jamieson T, Piovesan C, Shaw J. Virtual health care is having its moment. Rules will be needed. Healthy Debate. URL: https://healthydebate.ca/opinions/virtual-health-care-rules-covid [accessed 2022-12-08]
- INFOBulletin 21102: Virtual Care Services and Outside Use/Access Bonus. Ontario Ministry of Health. URL: https://www.health.gov.on.ca/en/pro/programs/ohip/bulletins/redux/bul211002.aspx [accessed 2022-12-08]
|ACG: adjusted clinical group|
|aOR: adjusted odds ratio|
|GEE: generalized estimating equation|
|RUB: resource utilization band|
|SMD: standardized mean difference|
Edited by R Kukafka; submitted 13.06.22; peer-reviewed by I Bennett, S Pesälä; comments to author 18.10.22; revised version received 31.10.22; accepted 01.12.22; published 12.01.23Copyright
©Lauren Lapointe-Shaw, Christine Salahub, Cherryl Bird, R Sacha Bhatia, Laura Desveaux, Richard H Glazier, Lindsay Hedden, Noah M Ivers, Danielle Martin, Yingbo Na, Sheryl Spithoff, Mina Tadrous, Tara Kiran. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 12.01.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.