Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 07.07.20 in Vol 22, No 7 (2020): July

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/18652, first published Mar 10, 2020.

This paper is in the following e-collection/theme issue:

    Original Paper

    Assessing Patient Experience and Healthcare Quality of Dental Care Using Patient Online Reviews in the United States: Mixed Methods Study

    1College of Dental Medicine, Western University of Health Sciences, Pomona, CA, United States

    2College of Health and Human Services, George Mason University, Fairfax, VA, United States

    3Arnold School of Public Health, University of South Carolina, Columbia, SC, United States

    *these authors contributed equally

    Corresponding Author:

    Chen Liang, PhD

    Arnold School of Public Health

    University of South Carolina

    Suite 347

    915 Greene Street

    Columbia, SC, 29208

    United States

    Phone: 1 803 777 8139

    Email: cliang@mailbox.sc.edu


    ABSTRACT

    Background: Over the last two decades, patient review websites have emerged as an essential online platform for doctor ratings and reviews. Recent studies suggested the significance of such websites as a data source for patients to choose doctors for healthcare providers to learn and improve from patient feedback and to foster a culture of trust and transparency between patients and healthcare providers. However, as compared to other medical specialties, studies of online patient reviews that focus on dentists in the United States remain absent.

    Objective: This study sought to understand to what extent online patient reviews can provide performance feedbacks that reflect dental care quality and patient experience.

    Methods: Using mixed informatics methods incorporating statistics, natural language processing, and domain expert evaluation, we analyzed the online patient reviews of 204,751 dentists extracted from HealthGrades with two specific aims. First, we examined the associations between patient ratings and a variety of dentist characteristics. Second, we identified topics from patient reviews that can be mapped to the national assessment of dental patient experience measured by the Patient Experience Measures from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Dental Plan Survey.

    Results: Higher ratings were associated with female dentists (t71881=2.45, P<.01, g=0.01), dentists at a younger age (F7, 107128=246.97, P<.001, g=0.11), and those whose patients experienced a short wait time (F4, 150055=10417.77, P<0.001, g=0.18). We also identified several topics that corresponded to CAHPS measures, including discomfort (eg, painful/painless root canal or deep cleaning), and ethics (eg, high-pressure sales, and unnecessary dental work).

    Conclusions: These findings suggest that online patient reviews could be used as a data source for understanding the patient experience and healthcare quality in dentistry.

    J Med Internet Res 2020;22(7):e18652

    doi:10.2196/18652

    KEYWORDS



    Introduction

    Over the last few years, patient review websites have gained increasing interest among health consumers, academic communities, and healthcare providers [1,2]. A tremendous amount of online patient reviews were shared with the public through patient review websites, becoming a common source of information for patients choosing a doctor. Accordingly, a growing body of literature in online patient review studies has been developed using public health informatics methods to leverage the distribution and determinants of data for informing public health and public policy, coinciding with the Research Framework of Infodemiology and Infoveillance [3].

    Despite debates on whether patient-generated reviews would be useful to improve healthcare quality [4-6], online reviews could serve as a valuable data source for understanding patient experience and the patient-provider relationship, and present unique values in improving dental care. Online reviews play an increasingly important role in health consumers’ decision-making for choosing dental services [7,8]. A recent national survey reported that 59% of respondents recognized the importance of patient review websites, 35% selected a doctor based on good ratings, and 37% avoided selecting a doctor based on bad ratings. Nearly 90% of the respondents rated listings of accepted health insurance on review sites as high importance [2]. Online patient review studies also found strong associations between ratings and doctor characteristics, such as gender, age, years of practice, online presence, and medical education and training, with variations at specialties [1,9].

    Online reviews are also rich in data on the patient experience, a critical measure of healthcare quality [10]. The notion of patient experience includes the entire scope of interactions, from appointment scheduling and access to information to communications with clinicians, cost, and payment. Positive patient experience is associated with better prevention and treatment adherence, safer healthcare outcomes, and better utilization of healthcare resources [11-15]. Incorporating data from the patient experience is also consistent with the principles of patient-centered care [10], as well as the goal of public reporting and the performance-based payment model set by the Centers for Medicare and Medicaid Services (CMS) [16].

    Online patient reviews can inform better decision-making among patients and can be used to improve healthcare quality. As such, many studies on online patient reviews in a variety of medical specialties have been reported [1]. Nevertheless, analyses of online reviews of dentists are sparse. We only identified one study of online patient reviews in dentistry conducted in Germany [17]. In the United States, studies of dental patient reviews are missing. To address the literature gaps, we conducted a study with two aims. The first aim was to characterize the online reviews of dental patients in the United States by analyzing data extracted from patient review websites. In particular, we examined the association between dentist characteristics and patient ratings.

    Our second aim was to understand to what extent online reviews can inform assessments of the patient experience by identifying semantic mentions of the patient experience in patient reviews. Dental patient experience is traditionally assessed with the CAHPS (Consumer Assessment of Healthcare Providers and Systems) Dental Plan Survey, administered by CMS [18]. Using online reviews for patient experience assessment is feasible but challenging. For instance, information extraction from online patient reviews is labor-intensive because the reviews are written in free text. Content analysis frequently used in social science research could not be applied to this study, in which there were hundreds of thousands of reviews to assess. The identification of information relevant to the patient experience is also necessary to ensure research validity, yet there are no guidelines or empirical studies on how to identify patient experience content in online reviews. Patient reviews cover a variety of topics, but not all of them are related to the patient experience. For example, topics of patient satisfaction should not be identified as patient experience. The patient experience is comprised of the clinical encounters and has been frequently used as an indicator of patient safety [19], whereas patient satisfaction relates to whether patients’ expectations about healthcare are met [20]. To overcome these challenges, we incorporated natural language processing [21] and human expert evaluation to identify relevant topics from online patient reviews. Natural language processing has extensive applications in health informatics and medical internet research, in which it has vastly improved the efficiency of processing free-text data using advanced statistical methods and automated computing [3]. Human expert evaluation mitigates the problem of misprocessed data typically produced by such a computerized method and improves the interoperability of the data analysis and results.


    Methods

    Data Acquisition

    Publicly accessible online review data were obtained from HealthGrades, a well-known patient review website in the United States. Among many other sites, we focused on HealthGrades for two reasons. First, HealthGrades is widely used by patients who receive healthcare services from a full range of medical specialties in the United States. Second, HealthGrades provides a well-organized sitemap structure that facilitates data extraction. We analyzed data from a single review site because the data structure and measures of dentist demographics and performance vary, which will hinder data consolidation. In addition, data from multiple patient review sites would have little impact on representation and generalizability of this study because an active dentist typically has profiles on all popular patient review sites.

    Online reviews for 204,751 dentists were extracted. This census approximates but does not fully match the workforce statistics (199,486 working dentists as of 2018) reported by the American Dental Association (ADA) [22] because some profiles are for recently inactive dentists and misclaimed or inappropriately captured profiles. The data contain the following attributes: state, city, specialty, gender, age, language, education, number of reviews, ratings, reviews, and wait time.

    Data Preprocessing

    There were 41 dental service listings in HealthGrades. We categorized these services into the 10 dental specialties defined by the ADA, general dentistry, and others (ie, unidentified and miscellaneous), resulting in 8 specialties (ie, dental anesthesiology, endodontics, oral and maxillofacial pathology, oral and maxillofacial surgery, orthodontics and dentofacial orthopedics, pediatric dentistry, periodontics, and prosthodontics), general dentistry, and others for downstream analyses. Public health dentistry and oral and maxillofacial radiology were excluded as they have only one entry at most. The data extraction was completed in September 2019. The study was identified as a nonhuman study by the Institutional Review Board of the University of South Carolina.

    Statistical Analyses

    We employed statistical analyses to assess the associations between ratings and dentists’ characteristics using R Project for Statistical Computing. We used descriptive statistics to calculate proportions and mean distributions. Based on reported online reviews studies in general medical specialties, we hypothesized that (1) female dentists, (2) young dentists, and (3) short wait times would be associated with higher overall ratings. We also hypothesized that specialties are associated with overall ratings. An independent sample t-test was used to test whether ratings differ by gender. Analysis of Variance (ANOVA) was used to test whether ratings differ by specialties, age, and wait time, respectively. We used Hedges g to approximate the effect size because of unbalanced sample sizes in comparison groups.

    Text Mining

    Semi-automated natural language processing was used to identify concepts related to patient experience with limited human labor required. Figure 1 is a diagram of text mining procedures. We used Python for text processing and computation. Built on our pilot study of analyzing patient-generated reviews [23], we first extracted reviews from online sites, followed by standard data cleaning procedures, including tokenization and removal of stop words.

    We then calculated bigram and trigram collocations. Collocations are habitual expressions of multiple words. In this study, we ranked using the “likelihood ratio” method [24], top 200 bigram and trigram collocations, respectively, from reviews associated with every rating category (ie, 1, 2, 3, 4, 5).

    We observed a number of the collocations that are irrelevant to concepts of dental care and patient experience (eg, “phone call” and “many years ago”) but were still ranked top 200 by likelihood ratio. Therefore, two raters (YL and CL) independently picked collocations related to patient experience using a 4-point Likert scale (“definitely relevant,” “somewhat relevant,” “somewhat irrelevant,” “definitely irrelevant”). Inter-rater reliability was assessed using Cohen kappa. Two raters discussed on the collocations that received contrary opinions (relevant vs irrelevant) until a consensus was reached.

    Next, we mapped patient-experience-related concepts onto a total of 17 composite measures from the Patient Experience Measures from the CAHPS Dental Plan Survey. Table 1 shows the 17 measures categorized in three dimensions. Two reviewers (RS and JT), who are professors of dental medicine and licensed dentists, independently completed the mapping procedures. A consensus was reached after a discussion of the initial mapping results.

    Figure 1. Diagram of text mining procedures.
    View this figure
    Table 1. Composite measures from the Patient Experience Measures from the CAHPS Dental Plan Survey.
    View this table

    Results

    Descriptive Statistics

    Table 2 shows the demographics of these dentists. Among the collected data, 154.683 (75.55%) dentists received at least one rating or review. The number of reviews ranged from 1 to 1789. There were more male dentists than females with a percentage difference of 85.87%. The majority of dentists (78.55%) were general dentists, followed by dentists specialized in orthodontics and dentofacial orthopedics (9.23%), pediatrics (4.76%), and endodontics (3.81%). The majority of dentists (91.83%) did not specify any second languages other than English. Among those who indicated speaking a second language, Spanish, Hindi, Arabic, French, and Chinese were most common. There were 150,571 (73.54%) dentists who received an overall rating of ≥3 out of 5, and 50,068 (24.45%) of all dentists received no ratings.

    Table 2. Dentist demographics.
    View this table

    Inferential Statistics

    ANOVA showed no significant effect of specialties on ratings (F9, 154673=58.74, P=1.37, g=0.06). However, the average rating was higher for female dentists (M=4.58) than male dentists (M=4.57) (t71881=2.45, P<.01, g=0.01). See Figure 2.

    We also found a significant effect of age on ratings (F7, 107128=246.97, P<.001, g=0.11) with younger age associated with higher ratings. The Tukey Honestly Significant Difference (Tukey HDS) test showed a significant difference in ratings between each pair of age groups except “30-39” vs “Over 89” (P=.15, g=0.01), “Under 30” vs “30-39” (P=.10, g=0.01), “40-49” vs “Over 89” (P=.95, g=0), “50-59” vs “60-69” (P=1.00, g=0), “50-59” vs “Over 89” (P=1.00, g=0), “60-69” vs “Over 89” (P=1.00, g=0), and “80-89” vs “Over 89” (P=1.00, g=0). See Figure 3.

    There was also a significant effect of wait time on ratings (F4, 150055=10417.77, P<.001, g=0.26) with shorter wait times associated with higher ratings. Tukey HDS showed statistical significance in comparing the mean difference of ratings in each pair of wait times (all P<.001, g=[0.01, 0.18]). See Figure 4.

    Figure 2. Overall ratings by gender.
    View this figure
    Figure 3. Overall ratings by age group.
    View this figure
    Figure 4. Overall ratings by wait time.
    View this figure

    Identification of Concepts Relating to Dental Patient Experience

    Cohen kappa (equally weighted) was 0.95 between the two raters who independently identified patient experience-related words and phrases from 2000 automatically extracted collocations. After discussion, they identified 29 words and phrases, then two other reviewers of dental experts independently mapped the 29 words and phrases onto the 17 composite measures in the Patient Experience Measures from the CAHPS Dental Plan Survey. Figure 5 shows a map of links. Each composite measure has 2-10 representing words and phrases. Out of the three dimensions of the patient experience, there were more topics representing “care from dentists and staff” and “access to dental care” compared to “dental plan costs and services.” Patients were more likely to discuss their experience with dentists and staff than health insurance providers. There were eight words and phrases related to the patient experience that did not correspond to any composite measures. Some of these topics were specific to dental care. For example, discomfort (eg, painful/painless root canal or deep cleaning) is a common type of feedback from dental patients. Topics relating to ethics (eg, high-pressure sales and unnecessary dental work) have received little attention in dentistry but merits further research.

    Figure 5. Mapping of Patient Experience Measures from the CAHPS Dental Plan Survey and the words and phrases extracted from patient reviews.
    View this figure

    Discussion

    Principal Findings

    Over the last few years, researchers have begun a systematic analysis of online patient reviews. In the United States, several empirical studies investigating online reviews of general healthcare services and specialties are well documented, but such studies have not been performed in dentistry [1]. To the best of our knowledge, this study represents the first study of online reviews of dental care in the United States. In particular, this study demonstrated that online patient reviews are an essential data source for studying dental patient experience. This study also characterized patient feedback to dental care, which can inform dental care quality improvement.

    We found several factors associated with overall dentist ratings. In particular, female dentists were rated slightly higher than their male counterparts, although this should be interpreted with the understanding that the effect size is small (g=0.01), and the sample size was small for senior dentists. The gender and age differences we found in this study were similar to a study of online reviews of dentists in Germany [25] as well as reviews of surgeons [26,27], but such a statistical significance was not consistent in other studies. Earlier studies on patient-provider communication suggested that female healthcare providers engage in significantly more active partnership behaviors, positive talk, and building trust with patients [28]. Female providers were also found associated with lower mortality and readmission as compared to their male counterparts [29]. The literature on healthcare quality suggests that younger physicians in acute care hospitals report lower mortality rates [30]. Our data also showed that higher overall ratings were associated with shorter wait times (g=0.26). This finding has been reported and discussed in several studies of online patient reviews [1]. Patient review websites such as HealthGrades and RateMDs have included “wait time/punctuality” as a default measure for healthcare providers.

    The proliferation of patient review websites represents a wealth of patient-experience data, but these data remain understudied. In this study, we identified unstructured descriptions of patient experience using a method integrating quantitative text mining and qualitative human evaluation. Our method recognizes the role of automated textual data analytics in harnessing information from online reviews, consistent with other recent studies [31-37], although some researchers argued the limitations of text mining because it involves limited supervision by human experts [31]. To minimize the impact of this lack of human oversight, we incorporated human evaluation procedures into the automated natural language processing effort. Our findings showed that patient reviews covered a full range of topics measured by the Patient Experience Measures from the CAHPS Dental Plan Survey and demonstrated a high level of correlation. Among the eight topics not corresponding to any measures from the survey, some topics may have provided a nuanced view of dental care as compared to patient survey responses [36]. These findings suggest that online patient reviews can be used to assess patient experience during dental care.

    Limitations

    Our study has the following limitations. First, data from Healthgrades, like data from any other patient review website, may be incomplete and biased. Not all dentist profiles have been claimed. Although authentication is required for dentist profile information, inaccuracies may still exist. The overall ratings may be biased because patients who are happy with health services are more likely to leave ratings and reviews [2]. Second, although there are good reasons for using data from a single review site, this limitation may still weaken the reliability of the study. Repeated measures and follow up studies are needed to evaluate the findings of the this study. Third, it is challenging to differentiate the descriptions of the patient experience from patient satisfaction as patients often write about their satisfactory or unsatisfactory experience with emotions and personal preferences. In this study, we only analyzed words and phrases at a semantic level, while contextual information is limited. Third, text mining is efficient for big data analyses but falls short in domain-specific and context-based analyses, in which traditional qualitative approaches should be considered as an essential complement of text mining [31].

    Despite these limitations, this study analyzed an extensive dataset and found an association between dentists and online patient reviews. The thematic analysis also identified themes of patient experience similar to those of CAHPS, suggesting that online patient reviews can inform improved quality in dental care.

    Conclusions

    This study demonstrated that PORs are a potential data source that can supply rich performance data from the patient perspective, based on which assessments of dental care quality and the patient experience is feasible. We also identified several factors associated with dentists’ overall ratings, which could be used to inform dental care quality improvement.

    Acknowledgments

    We thank Dr Jenny S Tjahjono for the contribution to the expert evaluation. This study is supported by a seed grant from the University of South Carolina.

    Conflicts of Interest

    None declared.

    References

    1. Hong YA, Liang C, Radcliff TA, Wigfall LT, Street RL. What Do Patients Say About Doctors Online? A Systematic Review of Studies on Patient Online Reviews. J Med Internet Res 2019 Apr 08;21(4):e12521 [FREE Full text] [CrossRef] [Medline]
    2. Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. Public awareness, perception, and use of online physician rating sites. JAMA 2014 Feb 19;311(7):734-735. [CrossRef] [Medline]
    3. Eysenbach G. Infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the Internet. J Med Internet Res 2009;11(1):e11 [FREE Full text] [CrossRef] [Medline]
    4. Bacon N. Will doctor rating sites improve standards of care? Yes. BMJ 2009;338:b1030. [Medline]
    5. McCartney M. Will doctor rating sites improve the quality of care? No. BMJ 2009 Mar 17;338:b1033. [CrossRef] [Medline]
    6. Jain S. Googling ourselves--what physicians can learn from online rating sites. N Engl J Med 2010 Jan 07;362(1):6-7. [CrossRef] [Medline]
    7. Farley DO, Elliott MN, Short PF, Damiano P, Kanouse DE, Hays RD. Effect of CAHPS performance information on health plan choices by Iowa Medicaid beneficiaries. Med Care Res Rev 2002 Sep;59(3):319-336. [CrossRef] [Medline]
    8. Bader JD, Shugars DA. Dental care plan assessment using the CAHPS Dental Plan Survey. J Public Health Dent 2012;72(4):295-301. [CrossRef] [Medline]
    9. Emmert M, Meier F, Pisch F, Sander U. Physician choice making and characteristics associated with using physician-rating websites: cross-sectional study. J Med Internet Res 2013 Aug 28;15(8):e187 [FREE Full text] [CrossRef] [Medline]
    10. Committee on Quality of Health Care in America, Institute of Medicine Staff. Crossing the quality chasm: A new health system for the 21st century. Washington DC: National Academies Press; Jul 19, 2001.
    11. Anhang Price R, Elliott MN, Zaslavsky AM, Hays RD, Lehrman WG, Rybowski L, et al. Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev 2014 Oct;71(5):522-554 [FREE Full text] [CrossRef] [Medline]
    12. Browne K, Roseman D, Shaller D, Edgman-Levitan S. Analysis & commentary. Measuring patient experience as a strategy for improving primary care. Health Aff (Millwood) 2010 May;29(5):921-925. [CrossRef] [Medline]
    13. Davies E, Shaller D, Edgman-Levitan S, Safran DG, Oftedahl G, Sakowski J, et al. Evaluating the use of a modified CAHPS survey to support improvements in patient-centred care: lessons from a quality improvement collaborative. Health Expect 2008 Jun;11(2):160-176 [FREE Full text] [CrossRef] [Medline]
    14. Friedberg MW, SteelFisher GK, Karp M, Schneider EC. Physician groups' use of data from patient experience surveys. J Gen Intern Med 2011 May;26(5):498-504 [FREE Full text] [CrossRef] [Medline]
    15. Goldstein E, Cleary PD, Langwell KM, Zaslavsky AM, Heller A. Medicare Managed Care CAHPS: A Tool for Performance Improvement. Health Care Financ Rev 2001;22(3):101-107 [FREE Full text] [Medline]
    16. CMS Issues final rule for first year of hospital value-based purchasing program. Centers for Medicare & Medicaid Services. 2011.   URL: https:/​/www.​cms.gov/​newsroom/​fact-sheets/​cms-issues-final-rule-first-year-hospital-value-based-purchasing-program [accessed 2020-06-04]
    17. Emmert M, Halling F, Meier F. Evaluations of dentists on a German physician rating Website: an analysis of the ratings. J Med Internet Res 2015 Jan 12;17(1):e15 [FREE Full text] [CrossRef] [Medline]
    18. CAHPS Dental Plan Survey. Agency for Healthcare Research and Quality. 2011.   URL: https://www.ahrq.gov/cahps/surveys-guidance/dental/index.html [accessed 2020-06-04]
    19. Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open 2013 Jan 03;3(1):e001570 [FREE Full text] [CrossRef] [Medline]
    20. What Is Patient Experience? Agency for Healthcare Research and Quality. 2016.   URL: https://www.ahrq.gov/cahps/about-cahps/patient-experience/index.html [accessed 2020-06-04]
    21. Manning CD, Schutze H. Foundations of statistical natural language processing. Cambridge: MIT Press; 1999.
    22. Workforce. American Dental Association. 2019.   URL: https://www.ada.org/en/science-research/health-policy-institute/dental-statistics/workforce [accessed 2020-06-04]
    23. Lin Y, Hong S, Liang C. Semantic Analysis of Online Dentist Review: Toward Assessing Safety and Quality of Dental Care. 2019 Presented at: International Conference on Human-Computer Interaction; 2019; Orlando FL p. 488-494. [CrossRef]
    24. Dunning T. Accurate methods for the statistics of surprise and coincidence. Computational linguistics 1993;19(1):61-74. [CrossRef]
    25. Emmert M, Meier F. An analysis of online evaluations on a physician rating website: evidence from a German public reporting instrument. J Med Internet Res 2013 Aug 06;15(8):e157 [FREE Full text] [CrossRef] [Medline]
    26. Nwachukwu BU, Adjei J, Trehan SK, Chang B, Amoo-Achampong K, Nguyen JT, et al. Rating a Sports Medicine Surgeon's "Quality" in the Modern Era: an Analysis of Popular Physician Online Rating Websites. HSS J 2016 Oct;12(3):272-277 [FREE Full text] [CrossRef] [Medline]
    27. Kirkpatrick W, Abboudi J, Kim N, Medina J, Maltenfort M, Seigerman D, et al. An Assessment of Online Reviews of Hand Surgeons. Arch Bone Jt Surg 2017 May;5(3):139-144 [FREE Full text] [Medline]
    28. Roter D, Lipkin M, Korsgaard A. Sex differences in patients' and physicians' communication during primary care medical visits. Med Care 1991 Nov;29(11):1083-1093. [CrossRef] [Medline]
    29. Tsugawa Y, Jena AB, Figueroa JF, Orav EJ, Blumenthal DM, Jha AK. Comparison of Hospital Mortality and Readmission Rates for Medicare Patients Treated by Male vs Female Physicians. JAMA Intern Med 2017 Feb 01;177(2):206-213 [FREE Full text] [CrossRef] [Medline]
    30. Tsugawa Y, Newhouse JP, Zaslavsky AM, Blumenthal DM, Jena AB. Physician age and outcomes in elderly patients in hospital in the US: observational study. BMJ 2017 May 16;357:j1797 [FREE Full text] [CrossRef] [Medline]
    31. Greaves F, Ramirez-Cano D, Millett C, Darzi A, Donaldson L. Harnessing the cloud of patient experience: using social media to detect poor quality healthcare. BMJ Qual Saf 2013 Mar;22(3):251-255. [CrossRef] [Medline]
    32. Hao H, Zhang K. The Voice of Chinese Health Consumers: A Text Mining Approach to Web-Based Physician Reviews. J Med Internet Res 2016 May 10;18(5):e108 [FREE Full text] [CrossRef] [Medline]
    33. Wallace BC, Paul MJ, Sarkar U, Trikalinos TA, Dredze M. A large-scale quantitative analysis of latent factors and sentiment in online doctor reviews. J Am Med Inform Assoc 2014;21(6):1098-1103 [FREE Full text] [CrossRef] [Medline]
    34. Hao H. The development of online doctor reviews in China: an analysis of the largest online doctor review website in China. J Med Internet Res 2015 Jun 01;17(6):e134 [FREE Full text] [CrossRef] [Medline]
    35. Hao H, Zhang K, Wang W, Gao G. A tale of two countries: International comparison of online doctor reviews between China and the United States. Int J Med Inform 2017 Mar;99:37-44. [CrossRef] [Medline]
    36. Ranard BL, Werner RM, Antanavicius T, Schwartz HA, Smith RJ, Meisel ZF, et al. Yelp Reviews Of Hospital Care Can Supplement And Inform Traditional Surveys Of The Patient Experience Of Care. Health Aff (Millwood) 2016 Apr;35(4):697-705 [FREE Full text] [CrossRef] [Medline]
    37. Agarwal AK, Mahoney K, Lanza AL, Klinger EV, Asch DA, Fausti N, et al. Online Ratings of the Patient Experience: Emergency Departments Versus Urgent Care Centers. Ann Emerg Med 2019 Jun;73(6):631-638. [CrossRef] [Medline]


    Abbreviations

    ADA: American Dental Association
    ANOVA: Analysis of Variance
    CAHPS: Consumer Assessment of Healthcare Providers and Systems
    CMS: Centers for Medicare and Medicaid Services
    Tukey HDS: Tukey’s Honestly Significant Difference


    Edited by G Eysenbach; submitted 10.03.20; peer-reviewed by M Nomali, A Mavragani; comments to author 07.04.20; revised version received 21.04.20; accepted 14.05.20; published 07.07.20

    ©Ye Lin, Y Alicia Hong, Bradley S Henson, Robert D Stevenson, Simon Hong, Tianchu Lyu, Chen Liang. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 07.07.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.