Published on in Vol 15, No 2 (2013): February

Eight Questions About Physician-Rating Websites: A Systematic Review

Eight Questions About Physician-Rating Websites: A Systematic Review

Eight Questions About Physician-Rating Websites: A Systematic Review

Authors of this article:

Martin Emmert1 ;   Uwe Sander2 ;   Frank Pisch1


  1. Boylan A, Williams V, Powell J. Online patient feedback: a scoping review and stakeholder consultation to guide health policy. Journal of Health Services Research & Policy 2020;25(2):122 View
  2. Van de Belt T, Engelen L, Berben S, Teerenstra S, Samsom M, Schoonhoven L. Internet and Social Media For Health-Related Information and Communication in Health Care: Preferences of the Dutch General Population. Journal of Medical Internet Research 2013;15(10):e220 View
  3. Schulz P, Rothenfluh F. Influence of Health Literacy on Effects of Patient Rating Websites: Survey Study Using a Hypothetical Situation and Fictitious Doctors. Journal of Medical Internet Research 2020;22(4):e14134 View
  4. Taylor J, Pagliari C. Comprehensive scoping review of health research using social media data. BMJ Open 2018;8(12):e022931 View
  5. Emmert M, Halling F, Meier F. Evaluations of Dentists on a German Physician Rating Website: An Analysis of the Ratings. Journal of Medical Internet Research 2015;17(1):e15 View
  6. Li S, Lee-Won R, McKnight J. Effects of Online Physician Reviews and Physician Gender on Perceptions of Physician Skills and Primary Care Physician (PCP) Selection. Health Communication 2019;34(11):1250 View
  7. Vallo Hult H, Hansson A, Svensson L, Gellerstedt M. Flipped healthcare for better or worse. Health Informatics Journal 2019;25(3):587 View
  8. Frost C, Mesfin A. Online Reviews of Orthopedic Surgeons: An Emerging Trend. Orthopedics 2015;38(4) View
  9. Deng Z, Hong Z, Zhang W, Evans R, Chen Y. The Effect of Online Effort and Reputation of Physicians on Patients’ Choice: 3-Wave Data Analysis of China’s Good Doctor Website. Journal of Medical Internet Research 2019;21(3):e10170 View
  10. Zwijnenberg N, Hendriks M, Bloemendal E, Damman O, de Jong J, Delnoij D, Rademakers J. Patients’ Need for Tailored Comparative Health Care Information: A Qualitative Study on Choosing a Hospital. Journal of Medical Internet Research 2016;18(11):e297 View
  11. McLennan S. Quantitative Ratings and Narrative Comments on Swiss Physician Rating Websites: Frequency Analysis. Journal of Medical Internet Research 2019;21(7):e13816 View
  12. Powell J, Atherton H, Williams V, Mazanderani F, Dudhwala F, Woolgar S, Boylan A, Fleming J, Kirkpatrick S, Martin A, van Velthoven M, de Iongh A, Findlay D, Locock L, Ziebland S. Using online patient feedback to improve NHS services: the INQUIRE multimethod study. Health Services and Delivery Research 2019;7(38):1 View
  13. Emmert M, Sauter L, Jablonski L, Sander U, Taheri-Zadeh F. Do Physicians Respond to Web-Based Patient Ratings? An Analysis of Physicians’ Responses to More Than One Million Web-Based Ratings Over a Six-Year Period. Journal of Medical Internet Research 2017;19(7):e275 View
  14. Kendall L, Botello T. Internet Sabotage: Negative Online Reviews of Psychiatrists. Psychiatric Annals 2016;46(12):715 View
  15. Kranzbühler A, Kleijnen M, Verlegh P, Teerling M. When Similarity Beats Expertise—Differential Effects of Patient and Expert Ratings on Physician Choice: Field and Experimental Study. Journal of Medical Internet Research 2019;21(6):e12454 View
  16. Cloney M, Hopkins B, Shlobin N, Dahdaleh N. Online Ratings of Neurosurgeons: An Examination of Web Data and its Implications. Neurosurgery 2018;83(6):1143 View
  17. Kool R, Kleefstra S, Borghans I, Atsma F, van de Belt T. Influence of Intensified Supervision by Health Care Inspectorates on Online Patient Ratings of Hospitals: A Multilevel Study of More Than 43,000 Online Ratings. Journal of Medical Internet Research 2016;18(7):e198 View
  18. McGrath R, Priestley J, Zhou Y, Culligan P. The Validity of Online Patient Ratings of Physicians: Analysis of Physician Peer Reviews and Patient Ratings. Interactive Journal of Medical Research 2018;7(1):e8 View
  19. Bilello J, Patel S, Potluri V, Gill G, Bagherpour A. Characterizing the Online Presence of Interventional Radiologists: A Potential Marketing Opportunity. Cureus 2020 View
  20. Widmer R, Shepard M, Aase L, Wald J, Pruthi S, Timimi F. The Impact of Social Media on Negative Online Physician Reviews: an Observational Study in a Large, Academic, Multispecialty Practice. Journal of General Internal Medicine 2019;34(1):98 View
  21. Terlutter R, Bidmon S, Röttl J. Who Uses Physician-Rating Websites? Differences in Sociodemographic Variables, Psychographic Variables, and Health Status of Users and Nonusers of Physician-Rating Websites. Journal of Medical Internet Research 2014;16(3):e97 View
  22. Möhlhenrich S, Wurbs M, Modabber A, Wolf M, Huber F, Fritz U. Influence of physician evaluation portals on orthodontist selection by patients. Journal of Orofacial Orthopedics / Fortschritte der Kieferorthopädie 2018;79(6):403 View
  23. Ferrara S, Hopman W, Leveridge M. Diagnosis, Bedside Manner and Comment Style are Predictive Factors in Online Ratings of Urologists. Urology Practice 2014;1(3):117 View
  24. Liu J, Matelski J, Bell C. Scope, Breadth, and Differences in Online Physician Ratings Related to Geography, Specialty, and Year: Observational Retrospective Study. Journal of Medical Internet Research 2018;20(3):e76 View
  25. Kleefstra S, Zandbelt L, Borghans I, de Haes H, Kool R. Investigating the Potential Contribution of Patient Rating Sites to Hospital Supervision: Exploratory Results From an Interview Study in the Netherlands. Journal of Medical Internet Research 2016;18(7):e201 View
  26. Goshtasbi K, Lehrich B, Moshtaghi O, Abouzari M, Sahyouni R, Bagheri K, Moshtaghi A, Tajran S, Lee L, Lin H, Djalilian H. Patients’ Online Perception and Ratings of Neurotologists. Otology & Neurotology 2019;40(1):139 View
  27. Bidmon S, Elshiewy O, Terlutter R, Boztug Y. What Patients Value in Physicians: Analyzing Drivers of Patient Satisfaction Using Physician-Rating Website Data. Journal of Medical Internet Research 2020;22(2):e13830 View
  28. Holliday A, Kachalia A, Meyer G, Sequist T. Physician and Patient Views on Public Physician Rating Websites: A Cross-Sectional Study. Journal of General Internal Medicine 2017;32(6):626 View
  29. Hao H, Zhang K, Wang W, Gao G. A tale of two countries: International comparison of online doctor reviews between China and the United States. International Journal of Medical Informatics 2017;99:37 View
  30. Emmert M, Meszmer N, Schlesinger M. A cross-sectional study assessing the association between online ratings and clinical quality of care measures for US hospitals: results from an observational study. BMC Health Services Research 2018;18(1) View
  31. Zwier S. Webcare in healthcare: providers' responses to patients' online reviews. British Journal of Healthcare Management 2019;25(10):1 View
  32. Fischer S, Pelka S, Riedl R, Duregger C. Understanding patients’ decision-making strategies in hospital choice: Literature review and a call for experimental research. Cogent Psychology 2015;2(1):1116758 View
  33. Wiley M, Rivas R, Hristidis V. Provider attributes correlation analysis to their referral frequency and awards. BMC Health Services Research 2016;16(1) View
  34. Widmer R, Maurer M, Nayar V, Aase L, Wald J, Kotsenas A, Timimi F, Harper C, Pruthi S. Online Physician Reviews Do Not Reflect Patient Satisfaction Survey Responses. Mayo Clinic Proceedings 2018;93(4):453 View
  35. Bardach N, Hibbard J, Greaves F, Dudley R. Sources of Traffic and Visitors’ Preferences Regarding Online Public Reports of Quality: Web Analytics and Online Survey Results. Journal of Medical Internet Research 2015;17(5):e102 View
  36. Bernstein D, Mesfin A. Physician-Review Websites in Orthopaedic Surgery. JBJS Reviews 2020;8(3):e0158 View
  37. Burn M, Lintner D, Cosculluela P, Varner K, Liberman S, McCulloch P, Harris J. Physician Rating Scales Do Not Accurately Rate Physicians. Orthopedics 2018;41(4) View
  38. Verhoef L, Van de Belt T, Engelen L, Schoonhoven L, Kool R. Social Media and Rating Sites as Tools to Understanding Quality of Care: A Scoping Review. Journal of Medical Internet Research 2014;16(2):e56 View
  39. Obele C, Duszak R, Hawkins C, Rosenkrantz A. What Patients Think About Their Interventional Radiologists: Assessment Using a Leading Physician Ratings Website. Journal of the American College of Radiology 2017;14(5):609 View
  40. McLennan S, Strech D, Meyer A, Kahrass H. Public Awareness and Use of German Physician Ratings Websites: Cross-Sectional Survey of Four North German Cities. Journal of Medical Internet Research 2017;19(11):e387 View
  41. Emmert M, Meier F, Pisch F, Sander U. Physician Choice Making and Characteristics Associated With Using Physician-Rating Websites: Cross-Sectional Study. Journal of Medical Internet Research 2013;15(8):e187 View
  42. Gilbert K, Hawkins C, Hughes D, Patel K, Gogia N, Sekhar A, Duszak R. Physician Rating Websites: Do Radiologists Have an Online Presence?. Journal of the American College of Radiology 2015;12(8):867 View
  43. Weenink J, Kool R, Hesselink G, Bartels R, Westert G. Prevention of and dealing with poor performance: an interview study about how professional associations aim to support healthcare professionals. International Journal for Quality in Health Care 2017;29(6):838 View
  44. Okike K, Uhr N, Shin S, Xie K, Kim C, Funahashi T, Kanter M. A Comparison of Online Physician Ratings and Internal Patient-Submitted Ratings from a Large Healthcare System. Journal of General Internal Medicine 2019;34(11):2575 View
  45. Baunacke M, Groeben C, Borgmann H, Schneider A, Kliesch S, Huber J. Evaluation des Urologenportals. Der Urologe 2016;55(7):923 View
  46. Emmert M, Meszmer N, Sander U. Do Health Care Providers Use Online Patient Ratings to Improve the Quality of Care? Results From an Online-Based Cross-Sectional Study. Journal of Medical Internet Research 2016;18(9):e254 View
  47. Wang J, Chiu Y, Yu H, Hsu Y. Understanding a Nonlinear Causal Relationship Between Rewards and Physicians’ Contributions in Online Health Care Communities: Longitudinal Study. Journal of Medical Internet Research 2017;19(12):e427 View
  48. Patel S, Cain R, Neailey K, Hooberman L. General Practitioners’ Concerns About Online Patient Feedback: Findings From a Descriptive Exploratory Qualitative Study in England. Journal of Medical Internet Research 2015;17(12):e276 View
  49. Daskivich T, Luu M, Noah B, Fuller G, Anger J, Spiegel B. Differences in Online Consumer Ratings of Health Care Providers Across Medical, Surgical, and Allied Health Specialties: Observational Study of 212,933 Providers. Journal of Medical Internet Research 2018;20(5):e176 View
  50. Jack R, Burn M, McCulloch P, Liberman S, Varner K, Harris J. Does experience matter? A meta-analysis of physician rating websites of Orthopaedic Surgeons. MUSCULOSKELETAL SURGERY 2018;102(1):63 View
  51. Grabner-Kräuter S, Waiguny M. Insights Into the Impact of Online Physician Reviews on Patients’ Decision Making: Randomized Experiment. Journal of Medical Internet Research 2015;17(4):e93 View
  52. Rotman L, Alford E, Shank C, Dalgo C, Stetler W. Is There an Association Between Physician Review Websites and Press Ganey Survey Results in a Neurosurgical Outpatient Clinic?. World Neurosurgery 2019;132:e891 View
  53. Hao H. The Development of Online Doctor Reviews in China: An Analysis of the Largest Online Doctor Review Website in China. Journal of Medical Internet Research 2015;17(6):e134 View
  54. Jabeen F, Hamid Z, Abdul W, Ghouzali S, Khan A, Malik S, Shaukat Khan M, Nawaz S, Wang H. Anonymity-preserving Reputation Management System for health sector. PLOS ONE 2018;13(4):e0195021 View
  55. Yu J, Samuel L, Yalçin S, Sultan A, Kamath A. Patient-Recorded Physician Ratings: What Can We Learn From 11,527 Online Reviews of Orthopedic Surgeons?. The Journal of Arthroplasty 2020;35(6):S364 View
  56. Haffey S, Hopman W, Leveridge M. Physicians' Earnings Do Not Affect Their Online Ratings. Frontiers in Public Health 2020;8 View
  57. Chen J, Presson A, Zhang C, Ray D, Finlayson S, Glasgow R. Online physician review websites poorly correlate to a validated metric of patient satisfaction. Journal of Surgical Research 2018;227:1 View
  58. Hong Y, Liang C, Radcliff T, Wigfall L, Street R. What Do Patients Say About Doctors Online? A Systematic Review of Studies on Patient Online Reviews. Journal of Medical Internet Research 2019;21(4):e12521 View
  59. Patel S, Cain R, Neailey K, Hooberman L. Public Awareness, Usage, and Predictors for the Use of Doctor Rating Websites: Cross-Sectional Study in England. Journal of Medical Internet Research 2018;20(7):e243 View
  60. Shah A, Yan X, Shah S, Mamirkulova G. Mining patient opinion to evaluate the service quality in healthcare: a deep-learning approach. Journal of Ambient Intelligence and Humanized Computing 2020;11(7):2925 View
  61. McLennan S. Rejected Online Feedback From a Swiss Physician Rating Website Between 2008 and 2017: Analysis of 2352 Ratings. Journal of Medical Internet Research 2020;22(8):e18374 View
  62. Emmert M, Meier F, Heider A, Dürr C, Sander U. What do patients say about their physicians? An analysis of 3000 narrative comments posted on a German physician rating website. Health Policy 2014;118(1):66 View
  63. Samora J, Lifchez S, Blazar P. Physician-Rating Web Sites: Ethical Implications. The Journal of Hand Surgery 2016;41(1):104 View
  64. Lu S, Rui H. Can We Trust Online Physician Ratings? Evidence from Cardiac Surgeons in Florida. Management Science 2018;64(6):2557 View
  65. Patel S, Cain R, Neailey K, Hooberman L. Exploring Patients’ Views Toward Giving Web-Based Feedback and Ratings to General Practitioners in England: A Qualitative Descriptive Study. Journal of Medical Internet Research 2016;18(8):e217 View
  66. Smith M, Bednarz L, Nordby P, Fink J, Greenlee R, Bolt D, Magnan E. Increasing Consumer Engagement by Tailoring a Public Reporting Website on the Quality of Diabetes Care: A Qualitative Study. Journal of Medical Internet Research 2016;18(12):e332 View
  67. McLennan S, Strech D, Reimann S. Developments in the Frequency of Ratings and Evaluation Tendencies: A Review of German Physician Rating Websites. Journal of Medical Internet Research 2017;19(8):e299 View
  68. Bustos S, Kapoor T, Schechter L, Ciudad P, Forte A, Del Corral G, Manrique O. Impact of social media presence on online reviews among plastic surgeons who perform gender confirming surgeries. Journal of Plastic, Reconstructive & Aesthetic Surgery 2020;73(4):783 View
  69. Kordzadeh N. Investigating bias in the online physician reviews published on healthcare organizations' websites. Decision Support Systems 2019;118:70 View
  70. McLennan S. The Content and Nature of Narrative Comments on Swiss Physician Rating Websites: Analysis of 849 Comments. Journal of Medical Internet Research 2019;21(9):e14336 View
  71. McLennan S, Strech D, Kahrass H. Why are so few patients rating their physicians on German physician rating websites? A qualitative study. BMC Health Services Research 2018;18(1) View
  72. Velasco B, Chien B, Kwon J, Miller C. Online Ratings and Reviews of American Orthopaedic Foot and Ankle Surgeons. Foot & Ankle Specialist 2020;13(1):43 View
  73. Emmert M, Meier F. An Analysis of Online Evaluations on a Physician Rating Website: Evidence From a German Public Reporting Instrument. Journal of Medical Internet Research 2013;15(8):e157 View
  74. Lu S, Rui H. Can We Trust Online Physician Ratings? Evidence from Cardiac Surgeons in Florida. SSRN Electronic Journal 2014 View
  75. Wallace B, Paul M, Sarkar U, Trikalinos T, Dredze M. A large-scale quantitative analysis of latent factors and sentiment in online doctor reviews. Journal of the American Medical Informatics Association 2014;21(6):1098 View
  76. Liu J, Goldberg H, Lentz E, Matelski J, Alam A, Bell C. Association Between Web-Based Physician Ratings and Physician Disciplinary Convictions: Retrospective Observational Study. Journal of Medical Internet Research 2020;22(5):e16708 View
  77. Lewis P, Kobayashi E, Gupta S. An Online Review of Plastic Surgeons in Southern California. Annals of Plastic Surgery 2015;74(Supplement 1):S66 View
  78. Saifee D, Zheng Z, Bardhan I, Lahiri A. Are Online Reviews of Physicians Reliable Indicators of Clinical Outcomes? A Focus on Chronic Disease Management. Information Systems Research 2020;31(4):1282 View
  79. Zhang S, Wang J, Chiu Y, Hsu Y. Exploring Types of Information Sources Used When Choosing Doctors: Observational Study in an Online Health Care Community. Journal of Medical Internet Research 2020;22(9):e20910 View
  80. Lu S, Rui H. Can We Trust Online Physician Ratings? Evidence from Cardiac Surgeons in Florida. SSRN Electronic Journal 2014 View
  81. Zhao H, Luu M, Spiegel B, Daskivich T. Correlation of Online Physician Rating Subscores and Association With Overall Satisfaction: Observational Study of 212,933 Providers. Journal of Medical Internet Research 2020;22(10):e11258 View
  82. Chua J, Nguyen E, Risbud A, Soltanzadeh‐Zarandi S, Lee A, Jamshidi S, Bayginejad S, Abouzari M. Online Ratings and Perceptions of Pediatric Otolaryngologists. The Laryngoscope 2021;131(10):2356 View
  83. Allen R, Agarwal S, Trolice M. Cross-sectional analysis of online patient reviews of infertility care providers. F&S Reports 2020;1(3):282 View
  84. Wu Q, Tang L. What Satisfies Parents of Pediatric Patients in China: A Grounded Theory Building Analysis of Online Physician Reviews. Health Communication 2022;37(10):1329 View
  85. Sama A, Matichak D, Schiller N, Li D, Donnally C, Damodar D, Cole B. The impact of social media presence, age, and patient reported wait times on physician review websites for sports medicine surgeons. Journal of Clinical Orthopaedics and Trauma 2021;21:101502 View
  86. Mao B, Li C. A good dentist or not: understanding the role of storytelling in online dentist reviews. Online Information Review 2022;46(2):356 View
  87. Emmert M, McLennan S. One Decade of Online Patient Feedback: Longitudinal Analysis of Data From a German Physician Rating Website. Journal of Medical Internet Research 2021;23(7):e24229 View
  88. Kroon M, Park J. Negative reviews online: an exploratory analysis of patient complaints about dental services in Western Australia. Australian Dental Journal 2022;67(2):138 View
  89. Stanworth J, Hsu R, Stanworth P, Kemp J, Tzen R, Wu H. When Culture Matters: Using Compliments and Complaints to Define and Influence Chinese Patients’ Satisfaction. Health Communication 2024;39(1):136 View
  90. Jabeen F, Hamid Z, Akhunzada A, Abdul W, Ghouzali S. Trust and Reputation Management in Healthcare Systems: Taxonomy, Requirements and Open Issues. IEEE Access 2018;6:17246 View
  91. Guetz B, Bidmon S, Dür M. Awareness of and interaction with physician rating websites: A cross-sectional study in Austria. PLOS ONE 2022;17(12):e0278510 View
  92. Saifee D, Hudnall M, Raja U. Physician Gender, Patient Risk, and Web-Based Reviews: Longitudinal Study of the Relationship Between Physicians’ Gender and Their Web-Based Reviews. Journal of Medical Internet Research 2022;24(4):e31659 View
  93. Spiro G, Sommerfeld C, Fung K, Quimby A, Pulkki K, Fortin M, Nguyen L. Poor Online Patient Ratings of Otolaryngologists in the United States: What are Patients Saying?. Ear, Nose & Throat Journal 2023:014556132211501 View
  94. Li G, Han C, Liu P. Does Internet Use Affect Medical Decisions among Older Adults in China? Evidence from CHARLS. Healthcare 2021;10(1):60 View
  95. Hoang V, Parekh A, Sagers K, Call T, Howard S, Hoffman J, Lee D. Patient Utilization of Online Information and its Influence on Orthopedic Surgeon Selection: Cross-sectional Survey of Patient Beliefs and Behaviors. JMIR Formative Research 2022;6(1):e22586 View
  96. Donnell C, Iafrate L, Worthington S. The 'five star' fallacy: an analysis of online reviews and testimonials of dental practices in Northern England. British Dental Journal 2022 View
  97. Richman E, Ogbaudu E, Pollock J, Brinkman J, Moore M, Arthur J, Karlen J. Characterizing Negative Online Reviews of Pediatric Orthopaedic Surgeons. Journal of Pediatric Orthopaedics 2022;42(5):e533 View
  98. Schwarz S, Schaefer C. Arztbewertungsportale: Was können sie leisten?. Public Health Forum 2015;23(3):163 View
  99. Hsu Y, Duan R, Chiu Y, Wang J. Understanding the Inequality of Web Traffic and Engagement in Online Healthcare Communities. Frontiers in Public Health 2022;10 View
  100. Chekijian S, Li H, Fodeh S. Emergency care and the patient experience: Using sentiment analysis and topic modeling to understand the impact of the COVID-19 pandemic. Health and Technology 2021;11(5):1073 View
  101. Adams E, Louters M, Kocsuta V, Ganesh M, Jang A, Ansbro B, Thavaseelan S, Kielb S. Gender Disparities and Differences Among Urologists Included in Top Doctor Lists. Urology 2023;173:215 View
  102. Barnett J, Bjarnadóttir M, Anderson D, Chen C. Understanding Gender Biases and Differences in Web-Based Reviews of Sanctioned Physicians Through a Machine Learning Approach: Mixed Methods Study. JMIR Formative Research 2022;6(9):e34902 View
  103. Stanbouly D, Rahhal Z, Talis A, Stanbouly R, Baron M, Arce K, Chandra S. Assessing reviews of academic oral and maxillofacial surgeons within the US on Healthgrades. Oral and Maxillofacial Surgery 2023;28(1):323 View
  104. Guetz B, Bidmon S. The Credibility of Physician Rating Websites: A Systematic Literature Review. Health Policy 2023;132:104821 View
  105. Kersting J, Maoro F, Geierhos M. Towards comparable ratings: Exploring bias in German physician reviews. Data & Knowledge Engineering 2023;148:102235 View

Books/Policy Documents

  1. Gendreau J, Ho A, Pendharkar A, Sussman E, Desai A. Quality Spine Care. View
  2. Fischer S. Challenges and Opportunities in Health Care Management. View
  3. Bäumer F, Grote N, Kersting J, Geierhos M. Information and Software Technologies. View
  4. Bidmon S. Advances in Advertising Research X. View
  5. Payne S, Cerqueira A, Kulikowski J, Hategan A, Waters H. Humanism and Resilience in Residency Training. View
  6. Miller E, Hefele J, Nadash P. Global Encyclopedia of Public Administration, Public Policy, and Governance. View
  7. Bäumer F, Kersting J, Kuršelis V, Geierhos M. Information and Software Technologies. View
  8. Fischer S, Emmert M. Challenges and Opportunities in Health Care Management. View
  9. Geierhos M, Bäumer F, Schulze S, Klotz C. Modeling and Using Context. View
  10. Miller E, Hefele J, Nadash P. Global Encyclopedia of Public Administration, Public Policy, and Governance. View
  11. Kersting J, Geierhos M. Natural Language Processing in Artificial Intelligence—NLPinAI 2020. View
  12. Newman A. Malpractice and Liability in Psychiatry. View
  13. Miller E, Hefele J, Nadash P. Global Encyclopedia of Public Administration, Public Policy, and Governance. View
  14. Kersting J, Geierhos M. Data Management Technologies and Applications. View