JMIR Publications


We are scheduled to perform a server upgrade on Thursday, November 30, 2017 between 4 and 6 PM Eastern Time.

Please refrain from submitting support requests related to server downtime during this window.

Journal of Medical Internet Research

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 17.02.17 in Vol 19, No 2 (2017): February

This paper is in the following e-collection/theme issue:

    Original Paper

    Virtual Visits for Acute, Nonurgent Care: A Claims Analysis of Episode-Level Utilization

    1HealthCore, Inc, Wilmington, DE, United States

    2Anthem, Inc, Indianapolis, IN, United States

    Corresponding Author:

    Aliza S Gordon, MPH

    HealthCore, Inc

    123 Justison Street

    Suite 200

    Wilmington, DE, 19801

    United States

    Phone: 1 302 230 2007

    Fax:1 302 230 2020

    Email:


    ABSTRACT

    Background: Expansion of virtual health care—real-time video consultation with a physician via the Internet—will continue as use of mobile devices and patient demand for immediate, convenient access to care grow.

    Objective: The objective of the study is to analyze the care provided and the cost of virtual visits over a 3-week episode compared with in-person visits to retail health clinics (RHC), urgent care centers (UCC), emergency departments (ED), or primary care physicians (PCP) for acute, nonurgent conditions.

    Methods: A cross-sectional, retrospective analysis of claims from a large commercial health insurer was performed to compare care and cost of patients receiving care via virtual visits for a condition of interest (sinusitis, upper respiratory infection, urinary tract infection, conjunctivitis, bronchitis, pharyngitis, influenza, cough, dermatitis, digestive symptom, or ear pain) matched to those receiving care for similar conditions in other settings. An episode was defined as the index visit plus 3 weeks following. Patients were children and adults younger than 65 years of age without serious chronic conditions. Visits were classified according to the setting where the visit occurred. Care provided was assessed by follow-up outpatient visits, ED visits, or hospitalizations; laboratory tests or imaging performed; and antibiotic use after the initial visit. Episode costs included the cost of the initial visit, subsequent medical care, and pharmacy.

    Results: A total of 59,945 visits were included in the analysis (4635 virtual visits and 55,310 nonvirtual visits). Virtual visit episodes had similar follow-up outpatient visit rates (28.09%) as PCP (28.10%, P=.99) and RHC visits (28.59%, P=.51). During the episode, lab rates for virtual visits (12.56%) were lower than in-person locations (RHC: 36.79%, P<.001; UCC: 39.01%, P<.001; ED: 53.15%, P<.001; PCP: 37.40%, P<.001), and imaging rates for virtual visits (6.62%) were typically lower than in-person locations (RHC: 5.97%, P=.11; UCC: 8.77%, P<.001; ED: 43.06%, P<.001; PCP: 11.26%, P<.001). RHC, UCC, ED, and PCP were estimated to be $36, $153, $1735, and $162 more expensive than virtual visit episodes, respectively, including medical and pharmacy costs.

    Conclusions: Virtual care appears to be a low-cost alternative to care administered in other settings with lower testing rates. The similar follow-up rate suggests adequate clinical resolution and that patients are not using virtual visits as a first step before seeking in-person care.

    J Med Internet Res 2017;19(2):e35

    doi:10.2196/jmir.6783

    KEYWORDS



    Introduction

    Health care delivery is moving outside traditional settings of physician offices and emergency departments (EDs) into convenient quick-care sites [1] such as retail health clinics (RHCs) and urgent care centers (UCCs). In addition to these relatively established alternatives for nonurgent acute care [2,3], venues maximizing the latest technology are emerging. Telehealth includes both asynchronous structured e-visits and online synchronous live video visits (virtual visits). Structured e-visits refer to online communication (eg, inputting symptoms into a website) and provision of treatment plans over the Internet, while virtual visits feature real-time video consultation. By providing a means for patients to receive health care from any location at all hours, telehealth platforms expand patient access to medical care [1,4,5]. While the American College of Physicians supports telemedicine, its policy statement stresses the importance of maintaining the physician-patient relationship [6].

    Because telehealth and particularly virtual visits are relatively new care options, published literature is lacking, with previous studies largely focused on acceptability or patient characteristics rather than outcomes [7-11]. Quality issues are particularly concerning for telehealth, such as whether physicians can provide accurate diagnoses without hands-on physical examinations of patients, whether patients receive appropriate laboratory testing after the visit, and whether antibiotics are overprescribed [12-14]. There are also concerns about whether the visits are truly efficient—if additional follow-up visits occur as a result of unresolved symptoms or because patients used online visits as a first opinion before seeking care in an office setting, this would diminish the apparent cost-saving potential. The few studies that examined the quality of virtual visits found higher antibiotic use after virtual visits compared with in-person office visits [13]—including prescription of broad-spectrum antibiotics [14]—for acute respiratory infections where antibiotics are not generally recommended. Providers of virtual visits were less likely than those in offices to order diagnostic tests to determine whether cases of pharyngitis were bacterial or viral, which is considered the standard of care [13]. Using follow-up visits as a proxy measure of misdiagnoses or treatment failures, researchers found a similar rate among patients participating in structured e-visits as those going to an office [6,12]. While an initial telehealth visit (e-visit or virtual) is usually less costly than an in-person visit, few if any studies analyzed episode-level costs associated with virtual visits (including follow-up care), although 1 study of structured e-visits found treatment costs per episode of care to be lower than traditional settings [7].

    Virtual visits have recently become available through independent online health care delivery sites that are often covered by patients’ health plans [14]. Starting in 2014, virtual visits became available to the members in this study as a covered benefit, with patient copays similar to primary care physicians (PCP) office visits. After patients create an initial profile, they choose from a list of available providers licensed in their state of residence [15]. Physicians are reimbursed $49 per virtual visit. The service has become increasingly popular with members, who received up to 4000 virtual visits per month through this service in 2015. Patient satisfaction has been high, with a net promoter score of 65%, based on an exit survey of patients who used the service administered as part of the health plan’s virtual care program (personal communication, W Adamson).

    To expand our understanding of the care provided and costs associated with virtual health care, we examined the care for specific acute, nonurgent conditions (eg, colds, allergies, urinary tract infections) provided by physicians via a virtual visit platform. Care provided through virtual visits, including subsequent care during a 3-week follow-up period, was compared with care delivered in the RHC, UCC, ED, and PCP office settings. This study is unique in assessing care and costs of virtual visit episodes, in contrast to previous studies of telehealth costs that have assessed structured e-visits only or have not taken into account follow-up care after the initial visit.


    Methods

    Study Design

    This cross-sectional retrospective study used data from commercially insured members receiving virtual care matched to members receiving care for similar conditions in other settings. The claims-based dataset was derived from the HealthCore Integrated Research Database (HIRD), a large administrative claims database containing medical and pharmacy claims for 14 Anthem commercial health plans geographically dispersed across the United States. The patient sample was identified from claims with service dates during the study period, January 1, 2014, through May 11, 2015. Researchers had access to a limited dataset containing no patient identifiers. This study was conducted in full compliance with the Health Insurance Portability and Accountability Act. This study was nonexperimental and was exempt from investigational review board approval.

    The index date was defined as the date of the first outpatient or ED claim in a 3-week period for 11 of the most commonly diagnosed conditions through the telehealth platform: sinusitis, upper respiratory infections (URIs), urinary tract infections (UTIs), conjunctivitis, bronchitis, pharyngitis, influenza, cough, dermatitis, nausea/vomiting/diarrhea, and ear pain, based on International Classification of Diseases, Ninth Revision (ICD-9) diagnosis codes (Multimedia Appendix 1). Baseline patient characteristics were determined from claims during the 6 months prior to the index date.

    Selection Criteria

    The study included adults younger than 65 years of age and children who had health plan eligibility for at least 6 months before and 3 weeks after the index date. Patients with serious or expensive health conditions, defined by Deyo-Charlson Comorbidity Index (DCI) scores of greater than 2, or with cystic fibrosis, transplant, end-stage renal disease, HIV, hemophilia, stroke, or respiratory failure were excluded.

    Episode Identification

    Visits for conditions of interest were classified according to the setting where the visit occurred: virtual (identified by Current Procedural Terminology [CPT] code 99444 and tax ID, representing all covered telehealth visits), RHCs (identified by tax ID and National Provider Identifier [NPI] numbers), UCCs (identified by tax ID and NPI numbers; only large national UCC chains included), EDs (identified by revenue codes and CPT codes), and PCP offices (identified by CPT codes for outpatient evaluation and management visits with provider specialty noted as primary care, internal medicine, general medicine, or pediatrics).

    An episode was defined as the index visit plus 3 weeks following. If patients had 2 or more potential index visits less than 3 weeks apart, only the first visit was used to identify an episode. If patients had more than 1 visit on the same day, a hierarchy was used to determine the index visit (as opposed to follow-up visit). The hierarchy was virtual > RHC > PCP > UCC > ED; this order was chosen due to likelihood of patients going to a more “urgent” care location after a different option was tried if more than 1 location was visited in a single day.

    Patient Selection

    Members receiving care from RHC, UCC, ED, and PCP offices were matched to those with virtual visits in a 3:1 ratio for each location (to increase statistical power) on acute condition, quarter and year of index date, state/region of residence, and child (<18 years) or adult age group.

    Care and Cost Outcome Measures

    The primary outcome measures were care provided (utilization during and following the visit) and cost of care. The follow-up period for outcomes assessment was from the index date to 3 weeks after to allow sufficient time for most minor conditions to resolve [16]. The follow-up period for antibiotic fills was 3 days from the index date since most antibiotic fills occurred during this time and fills occurring later in the episode may have occurred after a follow-up visit at a different location.

    Care provided was assessed by subsequent medical care after the initial visit (ie, outpatient evaluation and management visit [follow-up visit], ED visit, or inpatient hospitalization), laboratory tests performed, imaging performed, and antibiotic fill rates and use of broad-spectrum antibiotics, for patients where pharmacy data were available. Allowed cost per episode included the cost of the initial visit, subsequent medical care, and pharmacy costs.

    All care and costs during the 3-week episode were included, not just those for care with the same diagnosis as the index visit, since it is difficult to determine whether subsequent care is related to the initial visit (eg, pneumonia can develop after a different infection).

    Statistical Analysis

    The outcome measures were analyzed to determine differences between virtual visits and other locations of care. A significance level of α<.05 (2-sided) comparing each location with virtual was considered for all analyses (P values compare each location to virtual). Frequencies were reported and χ2 tests were used for all differences in care patterns/utilization. General linear models with gamma distribution and log link were used to compare costs and were adjusted to account for differences in age category and common baseline comorbidities (see Table 2 for specific age categories and comorbidities). We did not adjust for prior costs or utilization since prior care seeking is related to future care seeking behavior and does not necessarily match with health status. Due to the large variation in and skewed nature of follow-up costs, a sensitivity analysis was performed using winsorized values at the 5% and 95% level. Statistical analyses were conducted using SAS version 9.4 (SAS Institute, Inc).


    Results

    Selected Visits

    A total of 4635 virtual and 55,310 in-person visits were included in the analysis (13,832 RHC; 13,757 UCC; 13,840 ED; 13,881 PCP; see Figure 1 and Table 1). Pharmacy data were available for 3182 virtual and 29,562 other visits (7518 RHC; 7188 UCC; 7227 ED; 7629 PCP). The condition mix in each group consisted of the proportion of virtual visits with each of the 11 conditions (Table 1). Sinusitis and URI were the most highly represented conditions, accounting for more than half of the sample.

    Figure 1. Attrition; number of virtual visits at each step.
    View this figure
    Table 1. Visits by diagnosis.
    View this table

    Baseline Patient Characteristics

    In the RHC, UCC, and ED groups, the highest proportion of patients were 18 to 34 years of age, whereas the highest proportion of virtual patients were 35 to 49 years, and 50 to 64 years in the PCP group (Table 2). The majority of patients in all groups were women and had a low disease burden, indicated by a DCI score of zero. Of those with comorbidities, the most common was hypertension. While still low, DCI score and common comorbidities were highest among the PCP group (12% of the PCP group had DCI score of 1 or 2, compared with 10% of virtual and 7% of other groups).

    Table 2. Baseline characteristics.
    View this table

    Care Comparisons

    Subsequent outpatient medical care after the initial visit was similar between virtual visits and other treatment settings. The percentage of follow-up visits within 3 weeks of the index visit, which is a potential indicator of misdiagnosis or treatment failure, was similar between the virtual (28.09%), RHC (28.59%; P=.54), and PCP groups (28.10%; P=.96; Table 3). While the UCC group had slightly fewer follow-up visits (25.62%; P<.001), the ED group had more (34.19%; P<.001). The virtual group had fewer ED visits within 3 weeks of the index visit (1.32%) compared with the UCC (2.68%; P<.001), ED (6.47%; P<.001), and PCP groups (1.84%; P=.02) but similar to the RHC group (1.61%; P=.14). The percentage of hospitalizations within 3 weeks followed a similar pattern, with the percentage for the virtual group (0.15%) lower than the UCC (0.41%; P=.01), ED (0.96%; P<.001), and PCP groups (0.37%; P=.02) and similar to the RHC group (0.28%; P=.12).

    Table 3. Care patterns.
    View this table

    Overall laboratory tests within 3 weeks of the index date (including during the initial visit for nonvirtual visits) were lower for the virtual group (12.56%) compared with RHC (36.79%; P<.001), UCC (39.01 %; P<.001), ED (53.15%; P<.001), and PCP (37.40%; P<.001; Table 3). Lab testing was particularly low in virtual episodes compared with other locations of care for pharyngitis and UTI, where testing for a bacterial infection is common. Overall imaging rates were similar between the virtual and RHC groups (6.62% vs 5.97%, respectively; P=.11), but much lower than ED (43.06%; P<.001), and somewhat lower than UCC (8.77%; P<.001) and PCP groups (11.26%; P<.001). While lab and imaging rates differed by condition, the pattern of lower rates of lab and imaging testing in virtual episodes (except similar rates to RHC imaging) was consistent across conditions (see Table 3).

    Overall antibiotic fills within 3 days for the 6 most commonly treated infections (excluding influenza) was somewhat higher in the virtual group (70.51%) compared with all other sites (RHC 64.18%; P<.001, UCC 67.94%; P=.02, ED 56.73%; P<.001, PCP 68.19%; P=.03; Table 3), although there was variation by infection type. Fill rates after virtual visits tended to follow more similar patterns to UCC (for conjunctivitis, URI, and UTI) and PCP (for conjunctivitis, sinusitis, and URI) than RHC (similarity to sinusitis only) and ED (similarity to bronchitis only). Antibiotic fills were substantially higher after virtual visits than all other locations for pharyngitis (virtual 74.3%, RHC 29.6%; P<.001, UCC 53.8%; P<.001, ED 46.4%; P<.001, PCP 53.7%; P<.001).

    Broad-spectrum antibiotics were used as first-line treatment in the virtual group (68.99%) at a similar rate to the PCP group (69.28%; P=.82), but more often than in the RHC (59.98%; P<.001), UCC (66.28%; P=.04), and ED groups (61.90%; P<.001).

    Cost Comparisons

    Total costs per episode were $36, $153, $1735, and $162 more expensive at RHC, UCC, ED, and PCP settings, respectively, compared with virtual visits (Tables 4 and 5).

    Table 4. Cost of retail health clinic and urgent care center visits compared with virtual visits, adjusted for age categories and baseline comorbidities.
    View this table
    Table 5. Cost of emergency department and primary care physician visits compared with virtual visits, adjusted for age categories and baseline comorbidities.
    View this table

    As expected, the adjusted mean cost of the initial visit was lower for the virtual group ($49) than for RHC ($74; P<.001), UCC ($134; P<.001), ED ($1404; P<.001) and PCP ($109; P<.001; Tables 4 and 5). Follow-up medical costs for the virtual group were similar to or lower than the costs for each of the other sites of care. In the virtual group, 61.47% of patients had no follow-up medical costs, compared with 58.82% RHC, 63.63% UCC, 52.14% ED, and 52.53% PCP (Figure 2). Follow-up costs exceeded $500 for 7.08% of patients in the virtual group, compared with 6.01% RHC, 6.59% UCC, 15.23% ED, and 9.04% PCP; follow-up costs exceeded $5000 for 0.54% in the virtual group compared with 0.74% RHC, 0.88% UCC, 2.11% ED, and 1.03% PCP. Adjusted average follow-up medical costs were similar between the virtual ($200) and RHC groups ($204; P=.62) but higher for the UCC ($266; P<.001), ED ($584; P<.001), and PCP groups ($288; P<.001). At $90, the adjusted average pharmacy cost for a virtual episode was similar to UCC ($92; P=.44) and ED ($86; P=.21) and somewhat lower than RHC ($97; P=.02) and PCP ($104; P<.001).

    While average episode costs differed by condition, they tended to follow a similar pattern of virtual visits having lower medical costs than care at other locations across conditions (Multimedia Appendix 2). Additionally, the sensitivity analysis with winsorized values showed consistent relative costs of care between virtual and the other locations as the original analysis, except for lower RHC and UCC follow-up medical costs, although total episode cost differences were consistent (Multimedia Appendix 3).

    Figure 2. Follow-up medical costs, unadjusted.
    View this figure

    Discussion

    Principal Findings

    This retrospective, real-world analysis demonstrated that care received through virtual visits for nonurgent conditions was comparable to that received in in-person health care settings. Patients receiving care through virtual visits had similar follow-up outpatient evaluation and management visit rates as patients using other locations. This finding suggests not only that patients using virtual visits had their health problems resolved at similar rates as patients treated at other locations but also that patients were not using virtual visits as a first step before seeking in-person care. Interestingly, follow-up visit rates for the virtual group mirrored patients’ self-reported resolution of symptoms. An informal survey administered as part of the health plan’s virtual care program found 79% of patients who used it reported complete resolution of their health care concerns (personal communication, W Adamson).

    Lab testing rates, both overall and at the individual diagnosis level, were lower during virtual visits episodes than all in-person settings. Lab testing may be high at in-person locations for some conditions where it may not be needed to confirm the patient’s diagnosis [17,18]. However, the large differences between lab testing rates for pharyngitis and UTI suggest that patients with virtual care visits may not receive testing for these conditions where differentiating between a viral and bacterial infection is important for treatment. Antibiotics for the infections most commonly diagnosed through virtual visits were prescribed significantly more often after virtual visits than any of the other in-person treatment settings. The difference between virtual visits and other settings was even greater for pharyngitis, presumably because streptococcal infection was not ruled out with a lab test. While the use of broad-spectrum antibiotics was similar between virtual visits and PCP, it was higher than the other in-person settings.

    Episodes for patients who sought care at any of the in-person settings were more expensive than similar episodes beginning with a virtual visit. In addition to the virtual visit itself being less expensive than in-person visits, follow-up medical costs were lower after virtual visits than all other locations except for RHCs. Some of the lower episode costs can be attributed to lower rates of ED or inpatient follow-up care in addition to lower laboratory and imaging rates during the episode.

    Limitations

    A unique strength of this study was the large database allowing for a 3:1 match of episode-based care received in a number of alternative settings, but the exclusive use of claims data introduced several limitations. The accuracy of the diagnosis in claims may be a particular concern for virtual visits, where it may be more difficult for providers to diagnose a condition without a physical examination or supporting laboratory tests. Such errors not only may lead to inaccuracies in cost comparisons, but may also affect care patterns. It is not possible to determine disease severity from a diagnosis code alone, so cases seen in the ED, for example, may have been more severe, requiring more treatment than an average case handled by a virtual visit. Furthermore, claims do not provide complete information on the reasons patients chose a specific site of care. While patients may have chosen the ED because they perceived it to be the most convenient option even for a minor illness, it is also possible they believed their condition was severe and required urgent medical attention. However, the conditions included in this study tend to be relatively minor and treatable in nonurgent settings. Additionally, patients who chose virtual visits may have differed from those who chose other treatment settings in terms of their perception of the urgency of their condition, their health literacy, or their level of comfort using computers [7,8]. The analysis did not take into account whether a patient had multiple diagnoses at a single visit, which may have led to a more costly visit or additional follow-up care. Follow-up visits and costs may or may not have been related to the original visit, and we were unable to determine conclusively if a complaint was resolved at the original visit, if the follow-up visit was part of appropriate care, or if the follow-up visit represented inappropriate care-seeking behavior on the part of the patient. Additionally, claims do not provide sufficient information to determine if virtual visits were used in situations when individuals would have otherwise waited to see if the problem resolved on its own. A recent study suggested that RHCs increased utilization for low-acuity visits due to their lower price or convenience [19]. However, whether this is good or bad is a matter of perspective, since while costs may increase, the alternative care options may allow patients who may not have otherwise received appropriate care to receive the care they needed. Hence, our study focused on the difference in cost between care options, rather than potential savings of introducing virtual visits into the market or the necessity of the visits.

    Comparison With Prior Studies

    The rate of antibiotic prescriptions for the conditions included here may warrant additional study. Based on current guidelines and Choosing Wisely recommendations [20-22], prescription rates may be higher than desired in a variety of care settings. The virtual visits were associated with somewhat higher rates of antibiotic prescriptions than other sites of care overall, including for conditions for which clinical guidelines typically do not recommend antibiotics as a first line of treatment [20-22], although results were mixed when considered by condition. In some cases, the higher antibiotic prescription rate may have been due to a lower rate of laboratory testing associated with virtual visits, since a bacterial versus viral diagnosis could not be confirmed by lab test during virtual visits (eg, pharyngitis, where lab test rates were particularly low for virtual visits compared to other locations). This finding is consistent with previous research, which found higher antibiotic prescribing rates for telehealth than for office visits, especially for pharyngitis, bronchitis, and UTIs [23-25]. Future telehealth programs, particularly those integrated with a medical home or used for patients with an already existing physician-patient relationship, may be able to develop workflows that incorporate lab testing and may help with antibiotic prescribing decisions.

    Conclusions

    Virtual visits are growing rapidly, and our results indicate they are inexpensive alternatives to acute care administered at other locations. Patients receiving care through virtual visits seemed to have adequate clinical resolution compared with patients receiving care elsewhere, based on follow-up visit rates. Patients did receive additional care, such as laboratory testing or imaging, presumably when needed. Virtual visits did not appear to add to the total amount of care received as part of a care episode, as patients did not often seek care through telehealth plus another site for the same condition.

    Expansion of virtual health care services is inevitable given the growing use of mobile devices, patient demand for immediate and convenient access to care, and the continuously growing demands on physicians’ time. The focus of further research on virtual health care should be about optimizing patient outcomes for conditions best suited for virtual visits and examining how virtual visits can be used by physicians who have an existing personal relationship with the patient.

    Acknowledgments

    The authors thank D Marc Cram and Dianna Hayden for programming support and Cheryl Jones for editorial assistance. DM Cram, D Hayden, and C Jones are employees of HealthCore, Inc.

    Funding for this study was provided by Anthem, Inc. The authors are solely responsible for the study design, data collection, and interpretation; approvals from Anthem were not required for submission.

    Conflicts of Interest

    A Gordon and A DeVries are employees of HealthCore, Inc, a wholly owned outcomes research subsidiary of Anthem, Inc. W Adamson is an employee of Anthem, Inc, who works on LiveHealth Online. The authors were not compensated for this study beyond their salaries.

    Multimedia Appendix 1

    International Classification of Diseases, Ninth Revision, diagnosis codes.

    XLSX File (Microsoft Excel File), 41KB

    Multimedia Appendix 2

    Adjusted mean index visit medical, follow-up medical, and pharmacy costs, by condition.

    XLSX File (Microsoft Excel File), 53KB

    Multimedia Appendix 3

    Comparing original and winsorized adjusted costs (sensitivity analysis).

    XLSX File (Microsoft Excel File), 9KB

    References

    1. Mehrotra A. The convenience revolution for treatment of low-acuity conditions. JAMA 2013 Jul 3;310(1):35-36 [FREE Full text] [CrossRef] [Medline]
    2. Ashwood JS, Reid RO, Setodji CM, Weber E, Gaynor M, Mehrotra A. Trends in retail clinic use among the commercially insured. Am J Manag Care 2011 Nov;17(11):e443-e448 [FREE Full text] [Medline]
    3. DeVries A, Li C, Oza M. Strategies to reduce nonurgent emergency department use: experience of a Northern Virginia Employer Group. Med Care 2013 Mar;51(3):224-230. [CrossRef] [Medline]
    4. Hersh WR, Wallace JA, Patterson PK, Shapiro SE, Kraemer DF, Eilers GM, et al. Telemedicine for the Medicare population: pediatric, obstetric, and clinician-indirect home interventions. Evid Rep Technol Assess (Summ) 2001 Aug(24 Suppl):1-32 [FREE Full text] [Medline]
    5. Uscher-Pines L, Mehrotra A. Analysis of Teladoc use seems to indicate expanded access to care for patients without prior connection to a provider. Health Aff (Millwood) 2014 Feb;33(2):258-264. [CrossRef] [Medline]
    6. Daniel H, Sulmasy LS, Health and Public Policy Committee of the American College of Physicians. Policy recommendations to guide the use of telemedicine in primary care settings: an American College of Physicians position paper. Ann Intern Med 2015 Nov 17;163(10):787-789. [CrossRef] [Medline]
    7. Albert SM, Shevchik GJ, Paone S, Martich GD. Internet-based medical visit and diagnosis for common medical problems: experience of first user cohort. Telemed J E Health 2011 May;17(4):304-308 [FREE Full text] [CrossRef] [Medline]
    8. Hickson R, Talbert J, Thornbury WC, Perin NR, Goodin AJ. Online medical care: the current state of “eVisits” in acute primary care delivery. Telemed J E Health 2015 Feb;21(2):90-96. [CrossRef] [Medline]
    9. Jung C, Padman R. Virtualized healthcare delivery: understanding users and their usage patterns of online medical consultations. Int J Med Inform 2014 Dec;83(12):901-914. [CrossRef] [Medline]
    10. Mehrotra A, Paone S, Martich GD, Albert SM, Shevchik GJ. Characteristics of patients who seek care via eVisits instead of office visits. Telemed J E Health 2013 Jul;19(7):515-519 [FREE Full text] [CrossRef] [Medline]
    11. Flodgren G, Rachas A, Farmer AJ, Inzitari M, Shepperd S. Interactive telemedicine: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2015 Sep 07(9):CD002098. [CrossRef] [Medline]
    12. Mehrotra A, Paone S, Martich GD, Albert SM, Shevchik GJ. A comparison of care at e-visits and physician office visits for sinusitis and urinary tract infection. JAMA Intern Med 2013 Jan 14;173(1):72-74 [FREE Full text] [CrossRef] [Medline]
    13. Uscher-Pines L, Mulcahy A, Cowling D, Hunter G, Burns R, Mehrotra A. Access and quality of care in direct-to-consumer telemedicine. Telemed J E Health 2016 Apr;22(4):282-287. [CrossRef] [Medline]
    14. Uscher-Pines L, Mulcahy A, Cowling D, Hunter G, Burns R, Mehrotra A. Antibiotic prescribing for acute respiratory infections in direct-to-consumer telemedicine visits. JAMA Intern Med 2015 Jul;175(7):1234-1235. [CrossRef] [Medline]
    15. LiveHealth Online. 2015.   URL: https://www.livehealthonline.com/ [accessed 2017-01-26] [WebCite Cache]
    16. Mehrotra A, Liu H, Adams JL, Wang MC, Lave JR, Thygeson NM, et al. Comparing costs and quality of care at retail clinics with that of other medical settings for 3 common illnesses. Ann Intern Med 2009 Sep 1;151(5):321-328 [FREE Full text] [Medline]
    17. Albert RH. Diagnosis and treatment of acute bronchitis. Am Fam Physician 2010 Dec 01;82(11):1345-1350 [FREE Full text] [Medline]
    18. Rosenfeld RM, Piccirillo JF, Chandrasekhar SS, Brook I, Kumar KA, Kramper M, et al. Clinical practice guideline (update): Adult Sinusitis Executive Summary. Otolaryngol Head Neck Surg 2015 Apr;152(4):598-609. [CrossRef] [Medline]
    19. Ashwood JS, Gaynor M, Setodji CM, Reid RO, Weber E, Mehrotra A. Retail clinic visits for low-acuity conditions increase utilization and spending. Health Aff (Millwood) 2016 Mar;35(3):449-455. [CrossRef] [Medline]
    20. American Academy of Family Physicians. Don't prescribe antibiotics for otitis media in children aged 2-12 years with non-severe symptoms where the observation option is reasonable. 2016. Choosing Wisely   URL: http:/​/www.​choosingwisely.org/​clinician-lists/​american-academy-family-physicians-antibiotics-for-otitis-media-in-children/​ [WebCite Cache]
    21. American Academy of Pediatrics. Antibiotics should not be used for apparent viral respiratory illnesses (sinusitis, pharyngitis, bronchitis). 2016. Choosing Wisely   URL: http:/​/www.​choosingwisely.org/​clinician-lists/​american-academy-pediatrics-antibiotics-for-children-with-viral-respiratory-illness [WebCite Cache]
    22. Infectious Diseases Society of America. Avoid prescribing antibiotics for upper respiratory infections. 2016. Choosing Wisely   URL: http:/​/www.​choosingwisely.org/​clinician-lists/​infectious-diseases-society-antbiotics-for-upper-respiratory-infections/​ [accessed 2016-09-21] [WebCite Cache]
    23. Harris AM, Hicks LA, Qaseem A, High Value Care Task Force of the American College of Physicians and the Centers for Disease Control and Prevention. Appropriate antibiotic use for acute respiratory tract infection in adults: advice for high-value care from the American College of Physicians and the Centers for Disease Control and Prevention. Ann Intern Med 2016 Mar 15;164(6):425-434. [CrossRef] [Medline]
    24. Wong DM, Blumberg DA, Lowe LG. Guidelines for the use of antibiotics in acute upper respiratory tract infections. Am Fam Physician 2006 Sep 15;74(6):956-966 [FREE Full text] [Medline]
    25. American Academy of Family Physicians. 2016. Recommendations by type: clinical practice guidelines   URL: http://www.aafp.org/patient-care/clinical-recommendations/clinical-practice-guidelines.html [WebCite Cache]


    Abbreviations

    CPT: Current Procedural Terminology
    DCI: Deyo-Charlson Comorbidity Index
    ED: emergency department
    HIRD: HealthCore Integrated Research Database
    ICD-9: International Classification of Diseases, Ninth Revision
    NPI: National Provider Identifier
    PCP: primary care physician
    RHC: retail health clinic
    UCC: urgent care center
    URI: upper respiratory infection
    UTI: urinary tract infection


    Edited by G Eysenbach; submitted 11.10.16; peer-reviewed by AJ Goodin, S Kurklinsky; comments to author 24.11.16; revised version received 03.01.17; accepted 21.01.17; published 17.02.17

    ©Aliza S Gordon, Wallace C Adamson, Andrea R DeVries. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 17.02.2017.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.