This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Funding changes in response to the COVID-19 pandemic supported the growth of direct-to-consumer virtual walk-in clinics in several countries. Little is known about patients who attend virtual walk-in clinics or how these clinics contribute to care continuity and subsequent health care use.
The objective of the present study was to describe the characteristics and measure the health care use of patients who attended virtual walk-in clinics compared to the general population and a subset that received any virtual family physician visit.
This was a retrospective, cross-sectional study in Ontario, Canada. Patients who had received a family physician visit at 1 of 13 selected virtual walk-in clinics from April 1 to December 31, 2020, were compared to Ontario residents who had any virtual family physician visit. The main outcome was postvisit health care use.
Virtual walk-in patients (n=132,168) had fewer comorbidities and lower previous health care use than Ontarians with any virtual family physician visit. Virtual walk-in patients were also less likely to have a subsequent in-person visit with the same physician (309/132,168, 0.2% vs 704,759/6,412,304, 11%; standardized mean difference [SMD] 0.48), more likely to have a subsequent virtual visit (40,030/132,168, 30.3% vs 1,403,778/6,412,304, 21.9%; SMD 0.19), and twice as likely to have an emergency department visit within 30 days (11,003/132,168, 8.3% vs 262,509/6,412,304, 4.1%; SMD 0.18), an effect that persisted after adjustment and across urban/rural resident groups.
Compared to Ontarians attending any family physician virtual visit, virtual walk-in patients were less likely to have a subsequent in-person physician visit and were more likely to visit the emergency department. These findings will inform policy makers aiming to ensure the integration of virtual visits with longitudinal primary care.
Virtual walk-in clinics provide direct-to-consumer video, phone, or text-based physician consultations, often through a mobile phone app, and typically do not have a physical location. Prior to COVID-19, virtual walk-in clinics ostensibly helped meet a primary care need for people without a family physician or those who could not access their physician in a timely way, including those in rural settings [
Despite these positive perceptions, there remain concerns about the quality of care provided through virtual visits in general, and in particular the care provided by large, corporate virtual walk-in clinics [
Although other studies have described the rapid expansion of virtual care, previous reports could not distinguish corporate virtual walk-in clinic visits from other virtual primary-care visits, including those with a patient’s own physician [
We conducted a retrospective, cross-sectional study of all Ontario residents and those who had encounters at any of 13 selected virtual walk-in clinics.
Ontario is Canada’s most populous province, with over 14.5 million residents. Provincial health insurance is provided without premiums or copayments to all citizens and permanent residents and covers emergency department visits, hospitalizations, and all medically necessary physician care. Most primary care is provided by family physicians, and nearly 80% of the population is enrolled to a family physician working in a patient enrollment model [
Prior to April 2020, use of an approved platform (the Ontario Telemedicine Network [
To recruit patient partners, we advertised through ICES (formerly known as the Institute for Clinical Evaluative Sciences) and selected 4 individuals with diverse backgrounds in gender, race, location, profession, and lesbian, gay, bisexual, transgender, queer/questioning, and other sexual identity (LGBTQ+) status. They also all had previous experience as patients at walk-in clinics. The patient partners, through meetings and email correspondence with the principal investigator, reviewed the analytic plan and contributed to results interpretation.
This study was approved by the Women’s College Hospital Research Ethics Board (REB 2020-0095-E).
Population-based health administrative data sets were linked using unique encoded identifiers and analyzed at ICES in Ontario, Canada (
We developed a noncomprehensive list of virtual walk-in clinics by searching business names obtained from a list of group billing numbers and corresponding group names provided by the Ontario Ministry of Health. We used this list to identify all groups with “virtual” or “tele” in their name, used Google to search for the identified names, and reviewed the clinic websites to determine which provided exclusively virtual care (ie, without the possibility of an in-person office visit with a physician). In addition, we used Google to search for the combined terms “Canada” or “Ontario” AND “virtual clinic” or “telemedicine,” identifying several other groups for inclusion for a total of 20 virtual-only walk-in clinics. We then restricted the list to groups that had active billing claims during the period from April 1, 2019, to December 31, 2020 (n=13).
We included all family physicians with at least 5 virtual billing claims under one of our included virtual walk-in clinics from April 1 to December 31, 2020. The comparison group was all family physicians with active billings during this time.
We selected all patients who received at least one family physician visit at 1 of the 13 included virtual walk-in clinics from April 1 to December 31, 2020. The comparison group was all Ontario residents with an active health card and a health care contact within the previous 8 years as of April 1, 2020. For measures of health care use, we restricted the Ontario population to those who had at least one virtual family physician visit from April 1 to December 31, 2020.
We report the following patient characteristics: age, sex, neighborhood income quintile, urban or rural residence [
For patients with more than one virtual walk-in clinic visit, we randomly selected one virtual walk-in clinic visit and excluded all others (
We report the frequencies of the top 10 most common medical diagnoses in each group. We also report whether virtual encounters were with a patient’s enrolling family physician, the encounter day of the week, and 30-day postvisit health care use, including repeat virtual visits, office visits, low-acuity emergency department visits (defined as a Canadian Triage and Acuity Scale score of 4 to 5 [
We compared the characteristics of physicians who provided a virtual walk-in clinic visit to all family physicians with active billings. We also compared virtual walk-in clinic patients to the general Ontario population and the subset of the population that received any virtual family physician visit. Finally, we stratified health care use variables by the patients’ urban/rural residence status (large urban, small urban, or rural), because this is known to be associated with rates of emergency department use [
To compare groups, we used standardized mean differences (SMDs) and considered differences greater than 10% (0.1) to be significant [
Analyses were executed in SAS (version 9.4; SAS Institute Inc).
From April 2019 to December 2020, the weekly volume of patients increased 2-fold (
Weekly count of unique patients and unique physicians working for selected virtual walk-in clinics (n=13) in Ontario from the week beginning April 1, 2019, to the week beginning December 27, 2020. New virtual billing codes were introduced on March 14, 2020.
Compared to the overall Ontario population of family physicians with active billing between April 1 and December 31, 2020 (N=14,825;
Virtual walk-in clinic physician characteristics compared to all active billing family physicians. Measured between April 1, 2020, and December 31, 2020.
Physician characteristics | Provided >5 virtual walk-in clinic visits (n=242) | All family physicians with active billing (N=14,825) | Standardized mean differencea | |
|
||||
|
Mean (SD) | 40.3 (10.9) | 49.3 (14.0) | 0.72 |
|
Median (IQR) | 37 (32-46) | 48 (38-59) | 0.75 |
|
||||
|
25-34 | 102 (42.1) | 2471 (16.7) | 0.58 |
|
35-49 | 95 (39.3) | 5285 (35.6) | 0.07 |
|
50-64 | 34 (14) | 4843 (32.7) | 0.45 |
|
≥65 | 11 (4.5) | 2226 (15) | 0.36 |
Physicians self-reporting female gender, n (%) | 119 (49.2) | 7112 (48) | 0.02 | |
|
||||
|
0-5 | 37 (15.3) | 764 (5.2) | 0.34 |
|
6-10 | 57 (23.6) | 2280 (15.4) | 0.21 |
|
11-20 | 32 (13.2) | 2939 (19.8) | 0.18 |
|
21-30 | 35 (14.5) | 3027 (20.4) | 0.16 |
|
≥31 | 20 (8.3) | 4446 (30) | 0.57 |
|
Missing | 61 (25.2) | 1369 (9.2) | 0.43 |
|
||||
|
Large urban | 202 (83.5) | 11,010 (74.3) | 0.23 |
|
Small urban | 25 (10.3) | 2303 (15.5) | 0.16 |
|
Rural | 9 (3.7) | 1043 (7) | 0.15 |
|
Missing | 6 (2.5) | 469 (3.2) | 0.04 |
|
||||
|
Enhanced fee-for-service | 58 (24) | 2753 (18.6) | 0.13 |
|
Capitation | 40 (16.5) | 4665 (31.5) | 0.36 |
|
Team-based | 0 (0) | 1824 (12.3) | 0.53 |
|
Fee-for-service (no enrollment) | 136 (56.2) | 4926 (33.2) | 0.47 |
|
Other | 8 (3.3) | 657 (4.4) | 0.06 |
Number of patients seen per day as virtual visits, median (IQR) | 12 (5-22) | 13 (7-22) | 0.05 |
aA standardized mean difference of at least 10% (0.1) was considered to indicate a significant difference.
Compared to the overall Ontario population, patients who attended a virtual walk-in clinic visit were more likely to be young adults and less likely to be children or older adults (
Virtual walk-in clinic patients were less likely to be enrolled to a family physician than the Ontario population (84,861/132,168, 64.2% vs 10,908,871/14,709,408, 74.2%; SMD 0.22) and had lower continuity of care (SMD 0.44). Less than 0.1% (64/132,168) of virtual walk-in visits were with the patient’s enrolling family physician.
Compared to all Ontarians who had any virtual family physician visit, virtual walk-in clinic patients had fewer comorbidities (73,526/132,168, 55.6% vs 3,207,972/6,412,304, 50% were “low”; SMD 0.11) and lower levels of previous health care use (38,584/132,168, 29.2% vs 1,358,312/6,412,304, 21.2% were “low”; SMD 0.19). They were also more likely to have their virtual visit on a Saturday or Sunday.
Patient characteristics for visits at virtual walk-in clinics compared to the Ontario population with any virtual family physician visit. Measured between April 1 and December 31, 2020.
Characteristics | Visited a virtual walk-in clinic (n=132,168) | Ontario population overall (N=14,709,408) | Standardized mean difference a | |
Age (years), mean (SD) | 38.8 (19.4) | 41.3 (23.0) | 0.12 | |
|
||||
|
<18 | 13,730 (10.4) | 2,761,674 (18.8) | 0.24 |
|
18-29 | 35,300 (26.7) | 2,217,008 (15.1) | 0.29 |
|
30-44 | 35,980 (27.2) | 3,020,751 (20.5) | 0.16 |
|
45-64 | 31,714 (24) | 4,065,422 (27.6) | 0.08 |
|
65-74 | 9744 (7.4) | 1,497,270 (10.2) | 0.10 |
|
≥75 | 5700 (4.3) | 1,147,283 (7.8) | 0.15 |
Female sex, n (%) | 79,246 (60) | 7,472,638 (50.8) | 0.18 | |
|
||||
|
Lowest (1) | 29,822 (22.6) | 2,890,652 (19.7) | 0.07 |
|
2 | 26,598 (20.1) | 2,887,125 (19.6) | 0.01 |
|
3 | 25,667 (19.4) | 2,966,912 (20.2) | 0.02 |
|
4 | 26,141 (19.8) | 2,970,860 (20.2) | 0.01 |
|
Highest (5) | 23,669 (17.9) | 2,968,321 (20.2) | 0.06 |
|
Missing | 271 (0.2) | 25,538 (0.2) | 0.01 |
Recent provincial insurance registrant (past 10 years), n (%) | 14,334 (10.8) | 1,352,790 (9.2) | 0.05 | |
|
||||
|
Large urban | 83,484 (63.2) | 10,758,196 (73.1) | 0.22 |
|
Small urban | 37,581 (28.4) | 2,781,005 (18.9) | 0.23 |
|
Rural | 9863 (7.5) | 1,025,197 (7) | 0.02 |
|
Missing | 1240 (0.9) | 145,010 (1) | 0 |
|
||||
|
Low (0-5) | 73,526 (55.6) | 3,207,972 (50) | 0.11 |
|
Moderate (6-9) | 39,883 (30.2) | 2,203,659 (34.4) | 0.09 |
|
High (≥10) | 18,759 (14.2) | 1,000,673 (15.6) | 0.04 |
|
||||
|
Low (0-2) | 38,584 (29.2) | 1,358,312 (21.2) | 0.19 |
|
Moderate (3) | 63,896 (48.3) | 3,389,803 (52.9) | 0.09 |
|
High (4-5) | 29,688 (22.5) | 1,664,189 (26) | 0.08 |
Enrolled to a family physician, n (%) | 84,861 (64.2) | 10,908,871 (74.2) | 0.22 | |
|
||||
|
Capitation | 35,159 (26.6) | 4,241,999 (28.8) | 0.05 |
|
Enhanced fee-for-service | 43,114 (32.6) | 4,412,144 (30.0) | 0.06 |
|
Team-based | 27,508 (20.8) | 3,588,850 (24.4) | 0.09 |
|
Other group | 499 (0.4) | 99,775 (0.7) | 0.04 |
|
Fee-for-service (no enrollment) | 18,083 (13.7) | 1,038,591 (7.1) | 0.22 |
|
No prior physician primary care | 7805 (5.9) | 1,328,049 (9) | 0.12 |
|
||||
|
Continuity (%), median (IQR) | 50 (30-83.3) | 75 (50-100) | 0.44 |
|
Missing (<2 visits), n (%) | 25,119 (19) | 4,160,139 (28.3) | 0.22 |
|
||||
|
Monday | 22,758 (17.2) | 1,291,840 (20.1) | 0.08 |
|
Tuesday | 23,089 (17.5) | 1,386,411 (21.6) | 0.10 |
|
Wednesday | 21,799 (16.5) | 1,195,007 (18.6) | 0.06 |
|
Thursday | 21,396 (16.2) | 1,292,295 (20.2) | 0.10 |
|
Friday | 20,539 (15.5) | 887,193 (13.8) | 0.05 |
|
Saturday | 12,227 (9.3) | 253,311 (4) | 0.21 |
|
Sunday | 10,360 (7.8) | 106,247 (1.7) | 0.29 |
Index virtual visit was with enrolling physician, n (%)b | 64 (0) | 3,949,998 (61.6) | 1.79 |
aA standardized mean difference of at least 10% (0.1) was considered to indicate a significant difference.
bFor all variables related to the index visit (including comorbidity count and health care utilization band), the Ontario population group was restricted to those with any virtual family physician visit from April 1 to December 31, 2020 (n=6,412,304).
Diagnoses at virtual walk-in clinic visits were similar to those for all Ontarians’ virtual family physician visits (
Top 10 diagnoses for virtual walk-in clinic visits and for the Ontario population with any virtual family physician visit in 2020. Measured between April 1, 2020, and December 31, 2020.
Diagnoses | Values, n (%) | ||
|
|||
|
Other ill-defined conditions | 13,837 (10.5) | |
|
Cystitis | 6430 (4.8) | |
|
Mental health | 5226 (4) | |
|
Acute nasopharyngitis, common cold | 4838 (3.7) | |
|
Coronavirus | 4031 (3.1) | |
|
Gastrointestinal symptomsa | 3598 (2.7) | |
|
Other disorders of the urinary tract | 3518 (2.7) | |
|
Essential, benign hypertension | 3415 (2.6) | |
|
Cellulitis, abscess | 2993 (2.3) | |
|
Family planning, contraceptive advice, advice on sterilization, abortion | 2778 (2.1) | |
|
|||
|
Mental healthb | 488,468 (7.6) | |
|
Other ill-defined conditions | 393,541 (6.1) | |
|
Essential, benign hypertension | 372,793 (5.8) | |
|
Diabetes mellitus, including complications | 330,292 (5.2) | |
|
Musculoskeletal symptoms other than back painc | 225,615 (3.5) | |
|
Gastrointestinal symptomsa | 202,921 (3.2) | |
|
Eczema, atopic dermatitis, neurodermatitis | 140,262 (2.2) | |
|
Disorders of lipid metabolism | 136,877 (2.1) | |
|
Acute nasopharyngitis, common cold | 135,912 (2.1) | |
|
Lumbar strain, lumbago, coccydynia, sciatica | 107,305 (1.7) |
aGastrointestinal symptoms included anorexia, nausea and vomiting, heartburn, dysphagia, hiccup, hematemesis, jaundice, ascites, abdominal pain, melena, and masses.
bMental health included anxiety, neurosis, hysteria, neurasthenia, obsessive compulsive neurosis, and reactive depression.
cMusculoskeletal symptoms other than back pain included leg cramps, leg pain, muscle pain, joint pain, arthralgia, joint swelling, and masses.
Patients of virtual walk-in clinics had more repeat virtual visits within 30 days than Ontarians with a virtual family physician visit (40,030/132,168, 30.3% vs 1,403,304/6,412,304, 21.9%; SMD 0.19;
After adjustment, those who received a virtual walk-in clinic visit remained more likely to have an emergency department visit within 30 days in all 3 urban/rural residence strata: large urban (adjusted odds ratio [aOR] 2.26, 95% CI 2.08-2.45), small urban (aOR 2.08, 95% CI 1.99-2.18), and rural locations (aOR 1.87, 95% CI 1.69-2.07).
Thirty-day postvisit health care use for virtual walk-in clinic patients compared to Ontario population with a virtual family physician visit. Measured between April 1, 2020, and December 31, 2020.
Measures of use within 30 days following the virtual visit | Visit to a virtual walk-in clinic |
Ontario population with virtual family physician visit in 2020 (n=6,412,304), n (%) | Standardized mean differencea |
At least one repeated virtual visit with |
40,030 (30.3) | 1,403,778 (21.9) | 0.19 |
At least one in-person visit with |
309 (0.2) | 704,759 (11) | 0.48 |
At least one in-person visit with |
5633 (4.3) | 584,993 (9.1) | 0.20 |
At least one in-person visit with |
15,441 (11.7) | 980,556 (15.3) | 0.11 |
At least one emergency department visit | 11,003 (8.3) | 262,509 (4.1) | 0.18 |
At least one low-acuity emergency department visit | 3517 (2.7) | 69,425 (1.1) | 0.12 |
At least one urgent hospitalization | 1178 (0.9) | 49,717 (0.8) | 0.01 |
aA standardized difference of at least 10% (0.1) was considered to indicate a significant difference.
We compared patient characteristics and outcomes from visits to 13 virtual walk-in clinics to all virtual family physician visits in the Ontario population. Virtual walk-in patients were younger, were more likely to be female, and had lower continuity of care than the general population; they also had lower previous health care use than Ontario residents with any virtual family physician visit. Compared to Ontarians attending any family physician virtual visit, virtual walk-in patients were more likely to have a repeat virtual visit and less likely to have an in-person visit in the subsequent 30 days. They were also significantly more likely to visit the emergency department, a finding that held true in big cities, small towns, and rural areas, even after adjustment for potential confounders.
Our findings highlight two areas of potential concern with virtual walk-in clinics. The first is the lack of continuity of patient/physician relationships, a limitation shared with regular walk-in clinics. This is almost certainly accompanied by a lack of informational continuity, as presently there are no incentives or even regulatory frameworks compelling a virtual (or nonvirtual) walk-in physician to share information with a patient’s usual provider. Easy access to a family physician outside existing primary care relationships should be weighed against the risks of low-continuity care [
The second major concern is the potential downstream consequences of a care model that operates without the possibility of a physical examination. Patients who have a virtual visit with their own family physician have more options for in-person follow-up. In the absence of a physical examination, physicians at virtual walk-in clinics may recommend that patients go to emergency departments to be examined. Alternatively, our finding of higher rates of emergency department visits among virtual walk-in clinic users could reflect the downstream consequences of an incorrect or delayed diagnosis. The absence of a physical examination also has the potential to negatively affect other dimensions of care quality [
Reports from the United States, United Kingdom, and Sweden have described virtual-visit users as more likely to be healthy young adults [
Like others [
Our 4 patient partners provided several reasons why virtual walk-in clinics may be attractive to patients. They indicated that virtual walk-in clinics are convenient and require no travel, do not require scheduling an appointment or going through a “gatekeeper” to care such as an office assistant, might be more efficient when a patient is seeking a prescription or mental health care, and also provide relative anonymity to patients seeking care if they see a different physician on each visit. For these and other reasons, patients appreciate having the choice to visit a virtual walk-in clinic.
Developing a policy landscape that favors an efficient use of virtual visits is an urgent priority for health insurers [
Our study has several limitations. First, there are likely many more physicians and patients participating in virtual walk-in clinic care; however, because they are either not linked to a group billing number or are privately paid [
The number of Ontario patients and family physicians participating in a sample of virtual walk-in clinics rose rapidly after COVID-19–related physician fee schedule changes. Our findings suggest that these visits were associated with increased emergency department use. To ensure virtual walk-in clinics contribute positively to health outcomes and health system efficiency, policy makers should prioritize regulations and billing changes that ensure the integration of virtual and in-person visits while promoting continuity of care.
Supplementary materials.
adjusted clinical group
adjusted odds ratio
generalized estimating equation
resource utilization band
standardized mean difference
We would like to thank our patient partners CB, Jerome Johnson, Krysta Nesbitt, and Patrick Roncal for their contributions to the discussion about the study findings. We thank Alexander Kopp from ICES for his guidance in data analysis and methodology. We also thank our knowledge users Ed Jess, David Kaplan, David Price, Sundeep Banwatt, and Carolyn Canfield for their contributions to results interpretation. This project is supported by the Canadian Institutes of Health Research (project grant 175285). LLS is supported by the University of Toronto Department of Medicine, the Toronto General Hospital Research Institute, the Women’s College Institute for Health System Solutions and Virtual Care, and the Peter Gilgan Centre for Women’s Cancers at Women’s College Hospital, in partnership with the Canadian Cancer Society. TK is the Fidani Chair of Improvement and Innovation at the University of Toronto. She is supported as a clinician scientist by the Department of Family and Community Medicine at St. Michael’s Hospital and the University of Toronto. NMI is supported by a Canada Research Chair in Implementation of Evidence-based Practice and a Clinician Scholar award from the Department of Family and Community Medicine at Women’s College Hospital and the University of Toronto. This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health (MOH) and the Ministry of Long-Term Care. Parts of this material are based on data and information compiled and provided by the MOH, the Canadian Institute for Health Information, and Ontario Health. The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred.
The data set from this study is held securely in coded form at ICES. While legal data sharing agreements between ICES and data providers (eg, health care organizations and government) prohibit ICES from making the data set publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at www.ices.on.ca/DAS (email: das@ices.on.ca). The full data set creation plan and underlying analytic code are available from the authors upon request, understanding that the computer programs may rely upon coding templates or macros that are unique to ICES and are therefore either inaccessible or may require modification.
None declared.