Background: Telehealth, the delivery of health care through telecommunication technology, has potential to address multiple health system concerns. Despite this potential, only 15% of pediatric primary care clinicians reported using telemedicine as of 2016, with the majority identifying inadequate payment for these services as the largest barrier to their adoption. The COVID-19 pandemic led to rapid changes in payment and regulations surrounding telehealth, enabling its integration into primary care pediatrics.
Objective: Due to limited use of telemedicine in primary care pediatrics prior to the COVID-19 pandemic, much is unknown about the role of telemedicine in pediatric primary care. To address this gap in knowledge, we examined the association between practice-level telemedicine use within a large pediatric primary care network and practice characteristics, telemedicine visit diagnoses, in-person visit volumes, child-level variations in telemedicine use, and clinician attitudes toward telemedicine.
Methods: We analyzed electronic health record data from 45 primary care practices and administered a clinician survey to practice clinicians. Practices were stratified into tertiles based on rates of telemedicine use (low, intermediate, high) per 1000 patients per week during a two-week period (April 19 to May 2, 2020). By practice tertile, we compared (1) practice characteristics, (2) telemedicine visit diagnoses, (3) rates of in-person visits to the office, urgent care, and the emergency department, (4) child-level variation in telemedicine use, and (5) clinician attitudes toward telemedicine across these practices.
Results: Across pediatric primary care practices, telemedicine visit rates ranged from 5 to 23 telemedicine visits per 1000 patients per week. Across all tertiles, the most frequent telemedicine visit diagnoses were mental health (28%-36% of visits) and dermatologic (15%-28%). Compared to low telemedicine use practices, high telemedicine use practices had fewer in-person office visits (10 vs 16 visits per 1000 patients per week, P=.005) but more total encounters overall (in-office and telemedicine: 28 vs 22 visits per 1000 patients per week, P=.006). Telemedicine use varied with child age, race and ethnicity, and recent preventive care; however, no significant interactions existed between these characteristics and practice-level telemedicine use. Finally, clinician attitudes regarding the usability and impact of telemedicine did not vary significantly across tertiles.
Conclusions: Across a network of pediatric practices, we identified significant practice-level variation in telemedicine use, with increased use associated with more varied telemedicine diagnoses, fewer in-person office visits, and increased overall primary care encounter volume. Thus, in the context of the pandemic, when underutilization of primary care was prevalent, higher practice-level telemedicine use supported pediatric primary care encounter volume closer to usual rates. Child-level telemedicine use differed by child age, race and ethnicity, and recent preventive care, building upon prior concerns about differences in access to telemedicine. However, increased practice-level use of telemedicine services was not associated with reduced or increased differences in use, suggesting that further work is needed to promote equitable access to primary care telemedicine.
Telehealth, the delivery of health care through telecommunication technology, has potential to address multiple health system concerns; it can alleviate physician workforce shortages, improve access to care, mitigate disparities in health care, control costs, and enhance communication between clinicians [- ]. Despite this potential, the uptake of telehealth among pediatric clinicians has largely remained outside of primary care pediatrics, with growth instead observed in mental health, subspecialty care, and direct-to-consumer telemedicine provided by clinicians outside of the medical home [ , ]. The American Academy of Pediatrics (AAP) cautions against pediatric telemedicine provided outside of the primary care office due to concerns about fragmentation of care, suboptimal care quality, and lack of integrated follow-up; however, the AAP supports integration of telehealth into primary care pediatrics within the patient-centered medical home [ , ].
Despite the AAP endorsement of telehealth within primary care, only 15% of pediatric primary care clinicians reported using telemedicine as of 2016, with the majority identifying inadequate payment for these services as the largest barrier . As of February 2020, all state Medicaid programs had payment provisions for live video telehealth services; however, only 19 states paid for telehealth services when the patient was located in their home, and only 5 states mandated payment parity with in-person visits [ ]. This situation was reflected in similar stipulations by commercial payers. With limited payment options for telehealth services, especially for patients located at home, the adoption of telemedicine was not financially viable for most pediatric primary care offices outside of integrated care delivery systems before March 2020 [ , ].
In March 2020, the COVID-19 pandemic precipitated a rapid need for increased telehealth services to safely deliver care while limiting the risk of exposure to contagion that is inherent in an in-person setting . The nationwide need for telehealth services led to rapid changes in payment and regulations surrounding telehealth delivery. Specifically, updated policies allowed patients to be located in their homes during a telemedicine visit and allowed use of widely available technology platforms to deliver telehealth by waiving penalties for Health Insurance Portability and Accountability Act (HIPAA) violations [ , ]. These policy changes, and the shifting perceptions of risk versus benefit of in-person and virtual care, enabled the sudden adoption of telemedicine within primary care practices across the country [ - ].
Thus, we are witnessing an acute surge in telemedicine use within pediatric primary care; however, much is unknown about the potential uses and impact of telehealth in pediatric primary care, given the prior rarity of this model of care. In this paper, we describe the experience of a large pediatric primary care network within the first two months of telemedicine use during the COVID-19 pandemic. Specifically, we aimed to identify high versus low telemedicine-using primary care practices and compare (1) practice characteristics, (2) telemedicine visit diagnoses, (3) rates of in-person visits to the office, urgent care, and the emergency department (ED), (4) child-level variation in telemedicine use, and (5) clinician attitudes toward telemedicine across these practices.
Context and Study Population
We performed a retrospective analysis of electronic health record (EHR) data from 45 practices within a large pediatric primary care network. These practices are certified as patient-centered medical homes by The Joint Commission; together, they provide care for approximately 315,000 children throughout Western Pennsylvania across 13 counties. All the practices shared one EHR, which offered embedded video visits through a patient portal. Some of these practices had briefly trialed a model of acute care telemedicine in 2015; however, none of the practices were offering telemedicine services at the start of the pandemic.
Local payers began to offer payment for telemedicine when the patient is at home on March 17, 2020 . On March 23, 2020, the first county-specific stay-at-home order was issued in Pennsylvania [ , ]. Several practice leads trialed multiple telemedicine platforms and workflows from March 18-20, 2020, with implementation strategies shared with all practices via videoconference on March 23, 2020. The network quality and safety leaders led collaborative learning videoconferences two to three times per week for the next two months with all physicians, advanced practice providers (APPs), and practice managers, sharing telemedicine best practices and discussing other COVID-19–related topics. Initial relaxation of the stay-at-home order for the largest metropolitan county occurred on May 15, 2020, with transition of the state to its “yellow phase.”
We obtained encounter data for all telemedicine and in-person visits between March 18 and May 2, 2020, from the EHR, and we identified patient panels for each practice, defined as all patients with one or more encounters at the practice in the prior two years. For each practice’s patient panel, patient demographics, date of last preventive visit, and counts of telemedicine, office, ED, and urgent care visits were obtained from the EHR. To complement this EHR-based evaluation, we also surveyed the primary care clinicians across the practice network regarding the usability, usefulness, and perception of patient and clinician experience of telemedicine.
EHR Data and Variables
For each practice in the network, the number of practice clinicians (doctors and APPs) and practice site locations were obtained from network records. Practice site counties were classified as rural or urban using the 2013 rural-urban continuum codes .
Telemedicine visits were identified using EHR encounter type codes. All completed telemedicine visits from March 18 to May 2, 2020, were included. For each telemedicine visit, we extracted the practice site, age of the child on the date of visit, and primary visit diagnosis.
Telemedicine visit primary diagnoses were categorized based on International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) diagnosis codes into 22 broad ICD-10-CM diagnostic categories corresponding to organ systems . For nonspecific categories (eg, “symptoms, signs, and abnormal clinical and laboratory findings, not elsewhere classified” (R00-R99)), we reviewed the subcategories and recategorized them into the relevant organ system. For example, the subcategory of “symptoms and signs involving the skin and subcutaneous tissue” (R20-R23) was grouped with “diseases of the skin and subcutaneous tissues” (L00-L99) into “skin and subcutaneous tissue diagnoses.”
To compare the volumes of telemedicine visits with other modes of care delivery, we extracted each practice’s volumes from the EHR, including in-person office, urgent care, and ED visits, as well as telephone encounters (excluding those with a telemedicine or in-person visit on the same day) during a 2-week window during the pandemic (April 19 to May 2, 2020).
Finally, for each child identified as part of a practice’s panel, we extracted age, race and ethnicity, health insurance (Medicaid vs commercial), and whether the child had a preventive visit within the prior 12 months. The child’s race and ethnicity were originally recorded in the EHR based on parent response during the child’s first visit with the practice. Across all practices, 82% of patients were identified as White non-Hispanic; therefore, analyses by patient race and ethnicity were limited to comparing children identified as White non-Hispanic to children identified as any other racial or ethnic identity, of which the majority identified as “other” (9%), non-Hispanic Black (8%), or Hispanic (1%).
To complement this primarily EHR-based analysis, clinicians in the primary care network were invited to participate in a web-based survey. The survey items examined the usability and usefulness of telemedicine through items modified from the Technology Usability Questionnaire, which is a validated survey incorporating questions from the Technology Acceptance Model, Telemedicine Satisfaction Questionnaire, and Post-Study System Usability Questionnaire and encompasses five subscales: usefulness, ease of use, effectiveness, reliability, and satisfaction, with all items using Likert scales . We added questions pertaining to the physician experience of telemedicine, including perceived impact on quality of care (informed by the Institute of Medicine’s six domains of quality), impact on job satisfaction (informed by self-determination theory), and perceived usefulness of telemedicine for different visit reasons [ , ]. The survey included 37 questions, and it is available in . Participants had the option to identify their practice or leave this item blank.
Clinicians were invited to participate in the anonymous web-based survey from April 28 to May 14, 2020, through a series of 4 emails. The timeframe was chosen to capture summative attitudes and experiences of clinicians coinciding with the end of the EHR-based analysis.
Identification of Low, Intermediate, and High Telemedicine Use Practices
For each practice, to account for variation in practice size, we determined the total number of telemedicine visits completed per week and divided the number of visits by the number of active patients in that practice to provide a standardized rate of telemedicine visits per 1000 patients. To categorize high versus low telemedicine use practices, the rate of telemedicine visits per 1000 patients per week was averaged for the 2-week period (April 19 to May 2, 2020) occurring after the first month of telemedicine implementation. This time frame was chosen to categorize practices at a time where telemedicine visit volume had stabilized so that the analysis could focus on practices with high versus low use (as opposed to early vs late adopters). The 45 practices were categorized into tertiles based on their telemedicine visit rates, which we labeled as low, intermediate, or high telemedicine use practices.
The analyses compared practice-level characteristics, telemedicine visit diagnoses, in-person visit volumes, variation in volume by patient characteristics, and clinician attitudes across telemedicine use tertiles. Across practice-level telemedicine use tertiles, we compared practice-level characteristics and in-person visits per 1000 patients per week using Kruskal-Wallis tests. We compared the percentage of telemedicine visits within each individual diagnosis category across tertiles using logistic regression.
For patients in each practice’s panel, we determined the percentage of children who had one or more telemedicine visits during the 2-week period of the focused analysis (April 19 to May 2, 2020) by specific child characteristics (age, race and ethnicity, insurance type, and receipt of a preventive visit in the prior year). First, to assess whether there was a significant difference in telemedicine use by child characteristics across all practices (regardless of practice-level telemedicine use), we used a child-level logistic regression model clustered by practice but with no tertile variable. To determine whether increased practice-level use of telemedicine altered the differences in telemedicine use by child characteristics, we assessed the significance of an interaction term between each child characteristic category and practice tertile within a series of child-level logistic regression models clustered by practice across all tertiles. We tested the significance of the interaction terms using Wald tests, and we present the results as adjusted percentages determined through these models.
Clinician survey responses were compared across tertiles using linear regression clustered by practice, excluding respondents who did not identify their practice. We also compared responses for respondents who did not identify their practice versus those who did, again using linear regression, and found no significant differences.
All analyses were conducted in Stata version 16.1 (StataCorp) with significance assessed using an alpha level of .05.
Approval and Ethical Considerations
This analysis was part of a quality improvement project aimed at improving pediatric primary care telemedicine delivery and was approved by the University of Pittsburgh Medical Center’s Quality Review Committee. Projects approved by this committee do not meet the definition of human subjects research and therefore do not require formal approval by an institutional review board.
Starting on March 23, 2020, the network underwent a rapid reduction of in-person office visit volumes and a simultaneous increase in telemedicine visit volumes (). Telemedicine visit volume reached a steady state three to four weeks after implementation.
High telemedicine use practices had more physicians in the practice (median 4) than low telemedicine use practices (median 3, P=.04); however, all other practice-level characteristics were similar ().
|Variable||Telemedicine use||P value|
|Practices, n (%)||15 (33)||15 (33)||15 (33)||N/Aa|
|Telemedicine visits per 1000 patients per week, range||5-9.6||9.7-14||15-23||N/A|
|Active patients in practice, range||1200-8300||2600-7800||2000-10400||.08|
|Physicians, median (IQR)||2 (1-6)||5 (2-10)||6 (4-10)||.04|
|APPsb, median (IQR)||2 (1-3)||2 (2-6)||2 (1-5)||.70|
|Percent of patient population insured by Medicaid, median (IQR)||61 (50-77)||62 (53-73)||79 (62-82)||.07|
|Practice site (rural/urban)c|
|Rural county, n (%)||4 (27)||4 (27)||3 (20)||.90|
aN/A: not applicable.
bAPPs: advanced practice providers.
cBased on US Department of Agriculture rural-urban continuum codes.
Telemedicine Visit Diagnoses
Across all tertiles, telemedicine visits were most common for mental health and skin/soft tissue–related diagnoses (). However, the percentage of visits in each of these diagnostic categories varied with practice-level telemedicine use. Visits with skin-related diagnoses, for example, comprised 17.5% of telemedicine visits (467/2661) at low telemedicine use practices, and 15.0% of visits (1435/9587) at high telemedicine use practices (P=.006). Although visits for skin-related diagnoses represented a smaller percentage of telemedicine visits at high telemedicine use practices, the number of skin-related visits per 1000 patients per week was higher at high telemedicine use practices compared to low telemedicine use practices (median 14 vs 7 telemedicine skin related visits per 1000 patients per week at high and low telemedicine use practices, respectively; P<.001). High telemedicine use practices had larger percentages of telemedicine visits devoted to respiratory (P<.001), ear/mastoid (P<.001), and genitourinary (P=.02) diagnoses than low telemedicine use practices ( ).
|Variable||Telemedicine use||P valuea|
|Telemedicine visits per 1000 patients per week, range||5-9.6||9.7-14||15-23||N/Ab|
|Number of practices, n (%)||15 (33.3)||15 (33.3)||15 (33.3)||N/A|
|Visit diagnosis categoryc (total visits), n (%)|
|Mental, behavioral, and neurodevelopmental diagnoses||969 (36.4)||1704 (31.9)||2724 (28.4)||<.001|
|Skin and subcutaneous tissue diagnoses||467 (17.5)||801 (15.0)||1435 (15.0)||.006|
|Respiratory system diagnoses||297 (11.2)||684 (12.8)||1395 (14.6)||<.001|
|Digestive system diagnoses||182 (6.8)||396 (7.4)||697 (7.3)||.60|
|Infectious and parasitic diagnoses||176 (6.6)||390 (7.3)||746 (7.8)||.04|
|Injury/poisoning||92 (4)||184 (3.4)||437 (4.6)||.001|
|General symptoms and signs||77 (3.5)||241 (4.5)||416 (4.3)||.01|
|Eye diagnoses||77 (3.5)||147 (2.8)||255 (2.7)||.50|
|Ear and mastoid process diagnoses||45 (1.7)||230 (4.3)||395 (4.1)||<.001|
|Genitourinary system diagnoses||45 (1.7)||123 (2.3)||240 (2.5)||.02|
|Otherd||234 (8.8)||435 (8.2)||847 (8.8)||.60|
aP values indicate significance of proportion of telemedicine visits within each individual diagnosis category compared across practice-level telemedicine use tertiles using logistic regression.
bN/A: not applicable.
cCategories based on codes in the International Classification of Diseases, Tenth Revision, Clinical Modification.
dDiagnostic categories with less than 2% of visits are represented in this “other” category; these included neoplasms and hematologic diagnoses, endocrine, nutritional and metabolic diseases, nervous system diagnoses, circulatory system diagnoses, musculoskeletal diagnoses , peripartum and perinatal diagnoses, congenital anomalies, and symptoms and signs not otherwise classified, as well as codes for special purposes, injuries, and contact with health services.
In-Person Visit Volume
Compared to low telemedicine use practices, practices with high telemedicine use had fewer in-person office visits (median 10 vs 16 in-person office visits per 1000 patients per week at high vs low telemedicine use practices, respectively; P=.005;). Practices with high telemedicine use also had slightly more ED visits (median 2 vs 1 ED visits per 1000 patients per week at high and low telemedicine use practices, respectively; P=.02) but similar urgent care visits. When accounting for both in-person and telemedicine office visits, high telemedicine use practices had more total primary care encounters per 1000 patients per week (median 28 vs 22 encounters, P=.006). Of note, among high telemedicine use practices, total primary care encounters (in-person and telemedicine) represented a median of 53% of the weekly volume in 2019. In contrast, among low telemedicine use practices, total primary care encounters represented a median of 46% of the 2019 weekly volume. Telephone encounters occurring separate from a visit were similar across tertiles.
|Telemedicine use||P valuea|
|Practices, n (%)||15 (33)||15 (33)||15 (33)||N/Ab|
|Primary care patient visits|
|Telemedicine visits per 1000 patients per week, range||5-9.6||9.7-14||15-23||N/A|
|In-person office visits per 1000 patients per week, median (IQR)||16 (12-18)||11 (7-14)||10 (8-12)||.005|
|All primary care encounters per 1000 patients per week, median (IQR)||22 (19-26)||23 (19-26)||28 (25-30)||.006|
|In-person patient visits outside of primary care, median (IQR)|
|Urgent care visits per 1000 patients per week||0.4 (0-1)||1 (0.5-2)||1 (0.8-2)||.10|
|Emergency department visits per 1000 patients per week||1 (0.8-1)||2 (1-2)||2 (1-2)||.02|
|Total encounters per 1000 patients per week outside of primary care||2 (1-3)||3 (2-4)||3 (2-4)||.008|
|Total encounters per 1000 patients per week||25 (20-28)||25 (24-29)||30 (28-33)||.003|
|Patient telephone calls to practice not associated with a visit, median (IQR)|
|Telephone management without a visit per 1000 patients per week||17 (8-22)||14 (10-16)||15 (7-20)||.80|
aP values indicate practice-level in-person visits per 1000 patients per week compared across practice-level telemedicine use tertiles using Kruskal-Wallis tests.
bN/A: not applicable.
Patient Characteristics Associated With Telemedicine Use
In logistic regression models without practice-level tertile designation, use of telemedicine varied significantly by child race and ethnicity (P<.001), child age (P<.001), and receipt of preventive care in the prior 12 months (P<.001) but did not vary by child insurance category (P=.40). However, the interaction terms between the practice tertile and each of these characteristics were not significant (). This indicates that although differences exist in the full sample for telemedicine use by child race and ethnicity, child age, and recency of a preventive visit, increasing practice-level use of telemedicine did not change the relationship between these patient characteristics and the likelihood of a telemedicine visit.
|Model with no interaction term||Model with tertile and telemedicine use interaction term|
|All practices||Low telemedicine use practices||Intermediate telemedicine use practices||High telemedicine|
|P for interaction term|
|Practices, n (%)||45 (100)||15 (33.3)||15 (33.3)||15 (33.3)||N/Aa|
|Telemedicine visits per 1000 patients per week, range||5-23||5-9.6||9.7-14||15-23||N/A|
|Children, n (%)||244,473 (100)||66,295 (27.1)||84,093 (34.4)||94,085 (38.5)||N/A|
|Adjusted percentage of children with one or more telemedicine visit, % (95% CI)|
|By child age (years)b||.25|
|0-2||2.3 (2-2.7)||1.6 (0.8-2.3)||2.2 (1.9-2.6)||3.1 (2.6-3.5)|
|3-5||1.3 (1.1-1.5)||1 (0.7-1.4)||1 (0.9-1.2)||1.7 (1.4-2)|
|6-11||1.6 (1.5-1.8)||1.2 (0.9-1.4)||1.6 (1.5-1.8)||2 (1.8-2.2)|
|12-17||1.6 (1.4-1.7)||1.1 (0.9-1.3)||1.6 (1.3-1.9)||1.9 (1.7-2)|
|Non-Hispanic Black, Hispanic, and other race and ethnicity categories||1.4 (1.2-1.6)||0.7 (0.3-1.1)||1.3 (1.2-1.5)||1.8 (1.6-2.1)|
|Non-Hispanic White||1.7 (1.6-1.9)||1.2(1-1.5)||1.7 (1.5-1.8)||2.1 (2-2.3)|
|By receipt of preventive care within the last year||.26|
|Received preventive care in the last year||1.8 (1.7-2)||1.3 (1-1.6)||1.7 (1.6-1.9)||2.3 (2.1-2.5)|
|Did not receive preventive care in the last year||1.1 (0.9-1.2)||0.7 (0.6-0.8)||1.1 (1-1.2)||1.3 (1.1-1.5)|
|By insurance typec||.2|
|Medicaid-insured children||1.7 (1.5-1.9)||1.2 (0.9-1.6)||1.6 (1.4-1.8)||2.3 (2-2.6)|
|Commercially insured children||1.6 (1.5-1.8)||1.1 (0.8-1.4)||1.6 (1.5-1.7)||2 (1.8-2.2)|
aN/A: not applicable.
bPatients over 18 years of age were excluded (n=20,424).
cChildren whose insurance type was identified as self-pay were excluded (n=9258).
Attitudes of Clinicians Toward Telemedicine
The survey was completed by 121 clinicians, including 88 who identified their practice and 33 who did not (34% response rate). Clinician attitudes regarding the usability, usefulness for child health, usefulness for clinician experience, or impact of telemedicine did not vary significantly across tertiles ().
|Variable||Clinicians at low telemedicine use practices||Clinicians at intermediate telemedicine use practices||Clinicians at high telemedicine use practices||P valuea|
|Telemedicine visits per 1000 patients per week, range||5-9.6||9.7-14||15-23||N/Ab|
|Clinicians, n (%)||21 (24)||35 (40)||32 (36)||N/A|
|Usability, mean (SD)c|
|Ease of use and learnability||5.8 (0.8)||5.4 (1.2)||5.9 (0.9)||.50|
|Effectiveness||4.9 (0.8)||4.4 (1.4)||4.5 (1.3)||.30|
|Reliability||2.3 (1.4)||2.5 (1.6)||2.2 (1.3)||.80|
|Satisfaction and future use||5 (1.1)||4.8 (1.7)||4.8 (1.3)||.70|
|Usefulness—child and population health, mean (SD)d|
|Timeliness of care||3.8 (0.7)||3.7 (1)||3.8 (0.9)||.98|
|Equity in access to care||4 (0.8)||3.9 (1)||3.8 (1.1)||.50|
|Family-centeredness of care||3.4 (0.7)||3 (1)||3.1 (1.1)||.30|
|Health of my patients||2.8 (0.7)||3 (0.9)||2.9 (0.9)||.90|
|Continuity of care||3 (0.9)||3.1 (1.2)||3.1 (1)||.80|
|Safety of my patients||2.6 (0.7)||3.1 (1.1)||3 (1)||.20|
|Usefulness—clinician experience, mean (SD)d|
|Financial health of my practice||2.8 (0.9)||2.8 (1)||2.6 (1.1)||.30|
|Sense of accomplishment from my work||2.7 (0.8)||2.6 (1)||2.2 (1)||.07|
|Satisfaction with how I spend my time||2.6 (1.1)||2.5 (1.1)||2.2 (1)||.20|
|Sense of connectedness with patients||2.3 (0.9)||2.6 (1.2)||2.3 (1.2)||.80|
|Suitability of telemedicine for specific reasons, mean (SD)e|
|Acute care||2.2 (0.5)||2.3 (0.5)||2.3 (0.5)||.70|
|Chronic disease management||2.6 (0.7)||2.6 (0.7)||2.5 (0.7)||.70|
|Preventive care||1.9 (0.9)||1.8 (0.8)||1.8 (0.8)||.80|
|Follow-up care||2.4 (0.5)||2.3 (0.7)||2.3 (0.6)||.80|
|Care coordination||3.1 (0.8)||3 (0.7)||2.8 (0.7)||.30|
|Mental health||3.2 (0.7)||3 (0.5)||3 (0.7)||.30|
aP values reported from clinician survey responses compared across tertiles using linear regression clustered by practice.
bN/A: not applicable.
cSurvey questions answered using a 7-point Likert scale, where 1 indicates “strongly disagree” and 7 indicates “strongly agree.”
dSurvey questions answered using a 5-point Likert scale, where 1 indicates “much worse,” 3 indicates “about the same,” and 5 indicates “much better.”
eSurvey questions answered using a 4-point Likert scale, where 1 indicates “never” and 4 indicates “always.”
Among a network of pediatric practices, we examined practice-level variation in use of telemedicine during the COVID-19 pandemic, allowing us to explore ongoing questions about the relationship between telemedicine use, receipt of care, and equity in receipt of care. We found that increased practice-level telemedicine use was associated with more physicians in the practice, more varied telemedicine encounter diagnoses, and fewer in-person office visits per 1000 patients per week.
While concerns exist that telemedicine may promote overutilization, we found that in the context of the pandemic, greater pediatric primary care practice-level telemedicine use did not result in overutilization but rather supported primary care encounter volume slightly closer to usual rates during a time when underutilization of primary care and other health care settings was prevalent [- ]. We also found that high telemedicine use practices had slightly more ED visits than low telemedicine use practices during the study period. Again, this finding must be interpreted in the setting of overall decreased health care encounters during this time, such that this slight increase could represent either a greater ability of telemedicine to triage children to emergent care during the COVID-19 pandemic or, alternatively, a slightly greater need for emergent care for patients in these practices. Another possibility is that high telemedicine use practices may have adhered more strictly to guidelines to reduce in-person office visits; that goal, rather than telemedicine itself, may have led to the slight increase in ED visits.
In addition to concerns about overutilization, concerns have been raised about whether increased telemedicine use will improve equity of access to care or exacerbate disparities in access due to the digital divide [, , ]. In our sample, children without a preventive visit in the past year and children who identified as races or ethnicities other than White non-Hispanic were less likely to have had a telemedicine visit, while telemedicine use did not vary by child insurance type. These differences in the overall population use of telemedicine build upon prior concerns about differences in access to broadband internet, patient portals, and telemedicine [ , - ]. However, the nonsignificant interaction terms indicate that increased practice-level use of telemedicine services during this specific period neither reduced nor increased these differences. In cases where telemedicine is intended to be a tool to improve equity in access to care, these results indicate that simply increasing the use of telemedicine may not be sufficient to ensure more equitable access. Indeed, telemedicine has the potential to exchange one set of barriers to care (transportation issues, time constraints, hidden costs of missing work) for another (need for internet access, device capability, computer literacy). For telemedicine to more effectively reduce disparities in access, it will be necessary to implement telemedicine in ways that more intentionally overcome barriers.
While increased use of telemedicine by practices did not translate into greater equity in telemedicine use for patients, it did translate into more varied telemedicine use based on visit diagnoses. Additionally, we observed an increase in the proportion of visits related to respiratory and ear, nose, and throat (ENT) symptoms. Concerns have been raised about assessments of ears in the absence of tele-otoscopy, which was not available for the studied visits . Given that clinicians in low versus high telemedicine use tertiles reported similar views about the suitability of telemedicine for acute care, increased respiratory and ENT diagnoses at high telemedicine use practices may reflect a stronger practice-wide commitment to reducing in-person visits due to the COVID-19 pandemic as opposed to increased comfort with caring for these diagnoses via telemedicine. For all practices, regardless of their use tertile, the most common telemedicine visit diagnosis categories were mental health and skin-related diagnoses. This differs from the most common diagnoses for visits by children to commercial direct-to-consumer telemedicine, where the most common visit diagnosis category was nose/sinus infections [ ]. In contrast, the most common diagnosis group among these primary care telemedicine visits was mental health, which accounted for approximately 30% of telemedicine visits across the full set of practices.
One key limitation of this EHR analysis is we cannot account for patient or parent preferences for or satisfaction with telemedicine use. Additionally, the racial and ethnic diversity within our sample was minimal, limiting analyses by race and ethnicity to a comparison of White non-Hispanic children and children identified as any other race or ethnicity. We also sought to compare visit volume by parental language; however, limited numbers of children with parental preference for non-English languages were identified in the relevant EHR field (0.5%). Although systems are in place to integrate ED and urgent care visit information into the EHR, ED and urgent care visits outside of our health system may still have been missed. However, we do not anticipate that this would result in any systematic bias across the practice tertiles given the large number of integrated ED and urgent care centers across the region. We also note that we focused this analysis on telemedicine visit diagnoses and volumes and did not examine quality of care or clinical outcomes. Finally, this analysis focused on telemedicine use within a specific context; therefore, the generalizability of the findings will need to be examined through other sources.
This study demonstrates that a large pediatric primary care network rapidly integrated use of telemedicine when given a favorable payment environment and public health mandates. The integration of telemedicine allowed continued contact with patients during the pandemic, largely for mental health care, with high practice-level telemedicine use allowing for more encounters with patients per week during a time where underutilization of primary care was common. Further work is needed to understand the sustainability of the pandemic-related surge in primary care telemedicine use and to identify ways to enhance the ability of telemedicine to improve access for those with access barriers.
KS is a postdoctoral scholar supported in part by a Health Resources and Services Administration NRSA for Primary Care Research Award (T32 HP22240). The funder had no role in the study design, analysis of data, manuscript preparation, or decision to submit for publication.
KS and KNR conceptualized and designed the study, completed all statistical analyses, drafted the initial manuscript, and reviewed and revised the manuscript. AH, DW, PS, and JI conceptualized the study, coordinated the data collection, and reviewed and revised the manuscript. JT and JS collected data and reviewed and revised the manuscript. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
Conflicts of Interest
Clinician survey.DOCX File , 19 KB
- Committee on Pediatric Workforce, Marcin JP, Rimsza ME, Moskowitz WB. The Use of Telemedicine to Address Access and Physician Workforce Shortages. Pediatrics 2015 Jul 29;136(1):202-209. [CrossRef] [Medline]
- Marcin JP, Shaikh U, Steinhorn RH. Addressing health disparities in rural communities using telehealth. Pediatr Res 2016 Jan 14;79(1-2):169-176. [CrossRef] [Medline]
- NEJM Catalyst. What is telehealth? NEJM Catalyst. 2018 Feb 01. URL: https://catalyst.nejm.org/doi/full/10.1056/CAT.18.0268 [accessed 2020-12-10]
- Ray KN, Shi Z, Poon SJ, Uscher-Pines L, Mehrotra A. Use of Commercial Direct-to-Consumer Telemedicine by Children. Acad Pediatr 2019 Aug;19(6):665-669 [FREE Full text] [CrossRef] [Medline]
- Barnett ML, Ray KN, Souza J, Mehrotra A. Trends in Telemedicine Use in a Large Commercially Insured Population, 2005-2017. JAMA 2018 Nov 27;320(20):2147-2149 [FREE Full text] [CrossRef] [Medline]
- Conners G, Kressly SJ, Perrin JM, Richerson JE, Sankrithi UM, Committee on Practice and Ambulatory Medicine, Committee on Pediatric Emergency Medicine, Section on Telehealth Care, Section on Emergency Medicine, Subcommittee on Urgent Care, Task Force on Pediatric Practice Change. Nonemergency Acute Care: When It's Not the Medical Home. Pediatrics 2017 May;139(5):e20170629 [FREE Full text] [CrossRef] [Medline]
- Sisk B, Alexander J, Bodnar C, Curfman A, Garber K, McSwain SD, et al. Pediatrician Attitudes Toward and Experiences With Telehealth Use: Results From a National Survey. Acad Pediatr 2020 Jul;20(5):628-635 [FREE Full text] [CrossRef] [Medline]
- State Telehealth Laws and Reimbursement Policies: A Comprehensive Scan of the 50 States and District of Columbia. Center for Connected Health Policy. URL: https://www.cchpca.org/sites/default/files/2020-10/CCHP%2050%20STATE%20REPORT%20FALL%202020%20FINAL.pdf [accessed 2020-12-10]
- Reed ME, Parikh R, Huang J, Ballard DW, Barr I, Wargon C. Real-Time Patient–Provider Video Telemedicine Integrated with Clinical Care. N Engl J Med 2018 Oct 11;379(15):1478-1479. [CrossRef]
- Keesara S, Jonas A, Schulman K. Covid-19 and Health Care’s Digital Revolution. N Engl J Med 2020 Jun 04;382(23):e82. [CrossRef]
- Notification of Enforcement Discretion for Telehealth Remote Communications During the COVID-19 Nationwide Public Health Emergency. US Department of Health and Human Services. URL: https://www.hhs.gov/hipaa/for-professionals/special-topics/emergency-preparedness/notification-enforcement-discretion-telehealth/index.html [accessed 2020-12-10]
- Medicare Telemedicine Health Care Provider Fact Sheet. Centers for Medicare & Medicaid Services. 2020 Mar 17. URL: https://www.cms.gov/newsroom/fact-sheets/medicare-telemedicine-health-care-provider-fact-sheet [accessed 2020-06-22]
- Mehrotra A, Ray L, Brockmeyer DM, Barnett ML, Bender JA. Rapidly Converting to “Virtual Practices”: Outpatient Care in the Era of Covid-19. NEJM Catalyst. 2020 Apr 01. URL: https://catalyst.nejm.org/doi/full/10.1056/CAT.20.0091 [accessed 2020-12-10]
- Mueller B. Telemedicine Arrives in the U.K.: ‘10 Years of Change in One Week’. The New York Times. 2020 Apr 04. URL: https://www.nytimes.com/2020/04/04/world/europe/telemedicine-uk-coronavirus.html [accessed 2020-12-10]
- Barney A, Buckelew S, Mesheriakova V, Raymond-Flesch M. The COVID-19 Pandemic and Rapid Implementation of Adolescent and Young Adult Telemedicine: Challenges and Opportunities for Innovation. J Adolesc Health 2020 Aug;67(2):164-171 [FREE Full text] [CrossRef] [Medline]
- Wood SM, White K, Peebles R, Pickel J, Alausa M, Mehringer J, et al. Outcomes of a Rapid Adolescent Telehealth Scale-Up During the COVID-19 Pandemic. J Adolesc Health 2020 Aug;67(2):172-178 [FREE Full text] [CrossRef] [Medline]
- Telemedicine Guidelines Related to COVID-19. Pennsylvania Department of Human Services. 2020 Mar. URL: https://www.dhs.pa.gov/providers/Quick-Tips/Documents/PROMISeQuickTip229.pdf [accessed 2020-07-21]
- Order of the Governor of the Commonwealth of Pennsylvania for individuals to stay at home. Commonwealth of Pennsylvania Office of the Governor. 2020 Mar 23. URL: https://www.governor.pa.gov/wp-content/uploads/2020/03/03.23.20-TWW-COVID-19-Stay-at-Home-Order.pdf [accessed 2020-12-10]
- Rural-Urban Continuum Codes. United States Department of Agriculture. 2013. URL: https://www.ers.usda.gov/data-products/rural-urban-continuum-codes.aspx [accessed 2020-12-10]
- ICD-10-CM Official Guidelines for Coding and Reporting FY2020. Centers for Medicare & Medicaid Services. URL: https://www.cdc.gov/nchs/data/icd/10cmguidelines-FY2020_final.pdf [accessed 2020-12-10]
- Parmanto B, Lewis AL, Graham KM, Bertolet MH. Development of the Telehealth Usability Questionnaire (TUQ). Int J Telerehabil 2016;8(1):3-10 [FREE Full text] [CrossRef] [Medline]
- Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; Aug 30, 2001:702-703.
- Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist 2000;55(1):68-78. [CrossRef]
- Santoli J, Lindley MC, DeSilva MB, Kharbanda EO, Daley MF, Galloway L, et al. Effects of the COVID-19 Pandemic on Routine Pediatric Vaccine Ordering and Administration - United States, 2020. MMWR Morb Mortal Wkly Rep 2020 May 15;69(19):591-593 [FREE Full text] [CrossRef] [Medline]
- Hartnett KP, Kite-Powell A, DeVies J, Coletta MA, Boehmer TK, Adjemian J, National Syndromic Surveillance Program Community of Practice. Impact of the COVID-19 Pandemic on Emergency Department Visits - United States, January 1, 2019-May 30, 2020. MMWR Morb Mortal Wkly Rep 2020 Jun 12;69(23):699-704 [FREE Full text] [CrossRef] [Medline]
- Mehrotra A, Chernew M, Linetsky D, Hatch H, Cutler D. The Impact of the COVID-19 Pandemic on Outpatient Visits: A Rebound Emerges. The Commonwealth Fund. 2020 May 19. URL: https://www.commonwealthfund.org/publications/2020/apr/impact-covid-19-outpatient-visits [accessed 2020-12-10]
- Garcia S, Albaghdadi MS, Meraj PM, Schmidt C, Garberich R, Jaffer FA, et al. Reduction in ST-Segment Elevation Cardiac Catheterization Laboratory Activations in the United States During COVID-19 Pandemic. J Am Coll Cardiol 2020 Jun 09;75(22):2871-2872. [CrossRef] [Medline]
- Uchino K, Kolikonda MK, Brown D, Kovi S, Collins D, Khawaja Z, et al. Decline in Stroke Presentations During COVID-19 Surge. Stroke 2020 Aug;51(8):2544-2547. [CrossRef]
- Velasquez D, Mehrotra A. Ensuring The Growth Of Telehealth During COVID-19 Does Not Exacerbate Disparities In Care. Health Affairs. URL: https://www.healthaffairs.org/do/10.1377/hblog20200505.591306/full/ [accessed 2020-12-10]
- López L, Green AR, Tan-McGrory A, King RS, Betancourt JR. Bridging the Digital Divide in Health Care: The Role of Health Information Technology in Addressing Racial and Ethnic Disparities. The Joint Commission Journal on Quality and Patient Safety 2011 Oct;37(10):437-445. [CrossRef]
- Wilcock AD, Rose S, Busch AB, Huskamp HA, Uscher-Pines L, Landon B, et al. Association Between Broadband Internet Availability and Telemedicine Use. JAMA Intern Med 2019 Jul 29;179(11):1580 [FREE Full text] [CrossRef] [Medline]
- Walker DM, Hefner JL, Fareed N, Huerta TR, McAlearney AS. Exploring the Digital Divide: Age and Race Disparities in Use of an Inpatient Portal. Telemed J E Health 2020 May;26(5):603-613. [CrossRef] [Medline]
- Drake C, Zhang Y, Chaiyachati KH, Polsky D. The Limitations of Poor Broadband Internet Access for Telemedicine Use in Rural America: An Observational Study. Ann Intern Med 2019 May 12;171(5):382. [CrossRef]
- Yoon H, Jang Y, Vaughan PW, Garcia M. Older Adults' Internet Use for Health Information: Digital Divide by Race/Ethnicity and Socioeconomic Status. J Appl Gerontol 2020 Jan 16;39(1):105-110. [CrossRef] [Medline]
- Gerber JS. Need an Antibiotic? There's an App for That. Pediatrics 2019 May 08;143(5):e20190631 [FREE Full text] [CrossRef] [Medline]
|AAP: American Academy of Pediatrics|
|APP: advanced practice provider|
|ED: emergency department|
|EHR: electronic health record|
|ENT: ear, nose, and throat|
|HIPAA: Health Insurance Portability and Accountability Act|
|ICD-10-CM: International Classification of Diseases, Tenth Revision, Clinical Modification|
Edited by G Eysenbach; submitted 19.09.20; peer-reviewed by C Jacob, E van der Velde; comments to author 27.11.20; revised version received 02.12.20; accepted 03.12.20; published 18.12.20Copyright
©Kelsey Schweiberger, Alejandro Hoberman, Jennifer Iagnemma, Pamela Schoemer, Joseph Squire, Jill Taormina, David Wolfson, Kristin N Ray. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 18.12.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.