This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Since the time of Web 2.0, more and more consumers have used online doctor reviews to rate their doctors or to look for a doctor. This phenomenon has received health care researchers’ attention worldwide, and many studies have been conducted on online doctor reviews in the United States and Europe. But no study has yet been done in China. Also, in China, without a mature primary care physician recommendation system, more and more Chinese consumers seek online doctor reviews to look for a good doctor for their health care concerns.
This study sought to examine the online doctor review practice in China, including addressing the following questions: (1) How many doctors and specialty areas are available for online review? (2) How many online reviews are there on those doctors? (3) What specialty area doctors are more likely to be reviewed or receive more reviews? (4) Are those reviews positive or negative?
This study explores an empirical dataset from Good Doctor website, haodf.com—the earliest and largest online doctor review and online health care community website in China—from 2006 to 2014, to examine the stated research questions by using descriptive statistics, binary logistic regression, and multivariate linear regression.
The dataset from the Good Doctor website contained 314,624 doctors across China and among them, 112,873 doctors received 731,543 quantitative reviews and 772,979 qualitative reviews as of April 11, 2014. On average, 37% of the doctors had been reviewed on the Good Doctor website. Gynecology-obstetrics-pediatrics doctors were most likely to be reviewed, with an odds ratio (OR) of 1.497 (95% CI 1.461-1.535), and internal medicine doctors were less likely to be reviewed, with an OR of 0.94 (95% CI 0.921-0.960), relative to the combined small specialty areas. Both traditional Chinese medicine doctors and surgeons were more likely to be reviewed than the combined small specialty areas, with an OR of 1.483 (95% CI 1.442-1.525) and an OR of 1.366 (95% CI 1.337-1.395), respectively. Quantitatively, traditional Chinese medicine doctors (
Like consumers in the United States and Europe, Chinese consumers have started to use online doctor reviews. Similar to previous research on other countries’ online doctor reviews, the online reviews in China covered almost every medical specialty, and most of the reviews were positive even though all of the reviewing procedures and the final available information were anonymous. The average number of reviews per rated doctor received in this dataset was 6, which was higher than that for doctors in the United States or Germany, probably because this dataset covered a longer time period than did the US or German dataset. But this number is still very small compared to any doctor’s real patient population, and it cannot represent the reality of that population. Also, since all the data used for analysis were from one single website, the data might be biased and might not be a representative national sample of China.
Online doctor reviews have been happening across the world since the Internet Web 2.0 came into use in the early 2000s, and they have attracted health care researchers’ attention about how these reviews have been used in different countries [
We can see that various studies regarding online doctor reviews, either based on secondary data on how many doctors have been reviewed online or based on the first-hand survey data on how patients look at those online doctor reviews, have been emerging in the United States and Europe, but there has been no study about whether Chinese consumers use the Internet to review their health care providers. Considering the fact that China has the largest population of Internet users in the world [
Without a mature primary care system in China, most Chinese consumers now largely have to self-refer to any health care provider they can afford or who they believe is good based on little to no information [
Thus, the purpose of this study is to examine the following research questions about the current status of online doctor reviews in China based on the Good Doctor website: (1) How many doctors and how many specialty areas are available to be rated online on the Good Doctor website? (2) Which medical specialties are most likely to be rated? (3) Which medical specialties receive more reviews? (4) How are the quantitative rating scores distributed? and (5) What are the developing trends of online doctor reviews on the Good Doctor website?
According to meetings with Mr Hang Wang, the founder of the Good Doctor website, the original purpose in establishing this website was to help Chinese consumers to find good and appropriate specialists for their health care problems based on online reviews, after he personally experienced difficulty in finding a good specialist doctor in Beijing. In 2006, the Good Doctor website was launched in Beijing, China, and for the first time an online doctor review system became available for Chinese consumers.
Since its establishment, the Good Doctor company staff has been manually collecting information on Chinese doctors and their hospitals by various means—in China, a majority of doctors work at public hospitals where they are employees and have a responsibility for both inpatient service and outpatient service. The staff collected information by visiting hospital campuses in person, making phone calls to hospitals, or searching hospital websites, for those hospitals that had them. The staff then posted the collected information about the doctors on the Good Doctor website for consumers to browse and review for free. They knew that they would not have a national database to rely on and, as a start-up, they had limited human resources. Therefore, their strategy was to start with the largest and most famous hospitals in Beijing and Shanghai, then gradually cover the remaining parts of China, since the most reputable Chinese hospitals are concentrated mostly in Beijing or Shanghai. According to Fudan University’s hospital ranking system, which was based on a peer-review system on hospitals’ medical practice, quality of care, and research [
Once a doctor’s information is posted on the Good Doctor website, patients can anonymously review those doctors online based on their inpatient or outpatient experiences with the doctor. There are two types of reviews on the Good Doctor website: one is a quantitative review with two measures,
Based on the application programming interface (API) provided by the Good Doctor website, this study collected data on 314,624 doctors and their associated 3091 hospitals from the website as of April 11, 2014. After data cleaning by removing the records with missing values or abnormal values, there were 731,316 quantitative reviews, including both
Based on the dataset from the Good Doctor website, there are nine major specialty areas and one specialty area called “others,” which groups all the uncommon, small specialty areas not listed separately on the Good Doctor website.
Specialty areas, number of doctors, and number of reviews from the Good Doctor website.
Specialty areas | Total doctors, n | Doctors receiving reviews, n (%) | Total reviews, n | Average reviews per rated doctor |
Cancer | 2781 | 1317 (47.36) | 7018 | 5.3 |
Traditional Chinese medicinea | 27,299 | 12,011 (44.00) | 85,649 | 7.1 |
Gynecology-obstetrics-pediatricsa | 38,099 | 16,506 (43.32) | 122,073 | 7.4 |
Infectious disease | 1122 | 483 (43.05) | 2869 | 5.9 |
Internal medicinea | 66,162 | 22,345 (33.77) | 96,892 | 4.3 |
Orthopedics | 1008 | 495 (49.11) | 3592 | 7.3 |
Others | 112,483 | 36,038 (32.04) | 226,823 | 6.3 |
Psychiatry | 2848 | 1050 (36.87) | 5800 | 5.5 |
Oral health | 5106 | 2671 (52.31) | 15,690 | 5.9 |
Surgerya | 57,716 | 24,708 (42.81) | 164,910 | 6.7 |
Total | 314,624 | 117,624 (37.39) | 731,316 | 6.2 |
aSpecialty area that is among the four top specialty areas, which have the most number of doctors.
Number of reviews for each specialty per year.
Year | Cancer | TCMa | G-OB-Pb | IDc | IMd | OPe | Others | Psychiatry | Oral health | Surgery |
2006 | 0 | 20 | 115 | 1 | 142 | 0 | 155 | 5 | 13 | 108 |
2007 | 387 | 3351 | 6745 | 143 | 6335 | 143 | 11,055 | 261 | 642 | 7912 |
2008 | 691 | 10,114 | 16,008 | 415 | 12,508 | 334 | 26,006 | 659 | 1392 | 17,177 |
2009 | 1000 | 12,930 | 18,619 | 454 | 13,336 | 476 | 33,003 | 809 | 1849 | 21,623 |
2010 | 790 | 10,349 | 13,788 | 309 | 10,081 | 361 | 25,393 | 655 | 1900 | 18,830 |
2011 | 1181 | 15,186 | 20,185 | 520 | 14,755 | 671 | 38,650 | 954 | 2571 | 27,702 |
2012 | 1365 | 17,676 | 22,491 | 531 | 18,568 | 846 | 43,997 | 1212 | 3207 | 33,057 |
2013 | 1292 | 13,561 | 19,033 | 388 | 16,536 | 605 | 38,705 | 968 | 3222 | 30,134 |
2014 | 312 | 2462 | 5089 | 108 | 4631 | 156 | 9859 | 277 | 894 | 8367 |
Total | 7018 | 85,649 | 122,073 | 2869 | 96,892 | 3592 | 226,823 | 5800 | 15,690 | 164,910 |
aTraditional Chinese medicine (TCM)
bGynecology-obstetrics-pedicatrics (G-OB-P)
cInfectious disease (ID)
dInternal medicine (IM)
eOrthopedics (OP)
In order to further examine the research question of which types of doctors are more likely to be rated, a binary logistic regression model was constructed as follows:
Logit (Ratedi) = Specialty Areai + Physician Titlei + Hospital Leveli + Beijingi + Shanghaii (1)
Ratedi equals 1 if doctor i has been rated, otherwise it is 0. Specialty Areai is a categorical variable which differentiates the four major specialty areas from the rest of the combined specialty areas, combined specialties. The combined specialties combined the other five specialty areas listed by the Good Doctor website—infectious disease, orthopedics, psychiatry, oral health, and cancer—with the “others” for concision. Physician Titlei is a categorical variable, too, which indicates doctor i’s technical title from one of the four levels that was discussed earlier. There are three levels of hospital grades—Level 3 is the highest with more beds, better equipment, more highly skilled doctors, etc. This model also controls for Beijing and Shanghai because higher-ranking hospitals are more concentrated in these two cities than in other areas in China.
The following model examines which specialty area doctors would receive more ratings by using a multivariate linear regression model:
Rating_counti = Specialty Areai + Physician Titlei + Hospital Leveli + Beijingi + Shanghaii + errori (2)
The dependent variable, Rating_counti , is the number of ratings doctor i received. The independent variables are similar to those in model (1) for doctor i’s specialty area, technical title, hospital level, and whether the hospital is in Beijing, Shanghai, or another area.
Regarding the first research question of how many doctors and how many specialty areas are available for review, from
Results from the binary logistic regression (n=314,624).
Effect (independent variable)a | Odds ratio point estimateb,c,d | 95% Wald CI |
Traditional Chinese medicine | 1.483 | 1.442-1.525 |
Gynecology-obstetrics-pediatrics | 1.497 | 1.461-1.535 |
Internal medicine | 0.940 | 0.921-0.960 |
Surgery | 1.366 | 1.337-1.395 |
Chief physician | 4.648 | 4.525-4.774 |
Associate physician | 2.592 | 2.526-2.661 |
Attending physician | 1.624 | 1.576-1.673 |
Level 3 hospital | 2.047 | 1.995-2.100 |
Level 2 hospital | 1.590 | 1.548-1.633 |
Beijing | 1.532 | 1.486-1.579 |
Shanghai | 2.102 | 2.035-2.172 |
a
bPseudo R2 = .115.
cThe dependent variable is
d5% significance level for all values.
Results for linear regression for doctors in different areas receiving reviews (n=117,624).
Independent variablea | Parameter coefficient estimateb,c,d | Standard error |
|
|
Intercept | 2.17 | 0.24 | 9.14 | <.001 |
Traditional Chinese medicine | 0.62 | 0.17 | 3.58 | <.001 |
Gynecology-obstetrics-pediatrics | 0.97 | 0.15 | 6.34 | <.001 |
Internal medicine | -2.44 | 0.14 | -17.69 | <.001 |
Surgery | 0.26 | 0.13 | 1.95 | .052 |
Chief physician | 6.43 | 0.19 | 33.59 | <.001 |
Associate physician | 2.54 | 0.19 | 13.29 | <.001 |
Attending physician | 0.93 | 0.22 | 4.22 | <.001 |
Level 3 hospital | 0.53 | 0.17 | 3.10 | .002 |
Level 2 hospital | -1.29 | 0.18 | -7.15 | <.001 |
Beijing | 3.23 | 0.18 | 18.01 | <.001 |
Shanghai | 5.37 | 0.18 | 29.28 | <.001 |
a
bAdjusted R2=.0353.
cThe dependent variable is the
d5% significance level for all values.
Based on
Distribution of the number of ratings across four major specialty areas (absolute number).
Distribution of the number of ratings across four major specialty areas (relative percentage).
Distribution of treatment effect ratings across four major specialty areas.
Distribution of bedside manner ratings across four major specialty areas.
Distribution of treatment effect ratings across small specialty areas.
Distribution of bedside manner ratings across small specialty areas.
Number of ratings over time among four major specialty areas.
Number of doctors rated over time among four major specialty areas.
Average number of ratings per rated doctor over time among four major specialty areas.
First, we should realize that this study examines a dataset from a single website in China. Compared to developed countries, such as the United States, the United Kingdom, or Germany, the disadvantage of a dataset from a developing country like China is a lack of an official database which could be accessed to obtain the number of doctors in each specialty area at either the hospital level, the provincial level, or the national level. Thus, it is difficult to stratify this dataset to make a nationally representative sample. But the current dataset is an empirical dataset from the earliest and the largest available online doctor review website in China, and the four major specialty areas’ percentage compositions are close to the national level aggregated data. Therefore, this empirical dataset will assist us to understand the current status of online doctor reviews in China.
The average review rate of all doctors over about 8 years on the Good Doctor website was 37.4%, which is close to the national review rate in Germany in 2012 of 37% [
On the Good Doctor website, 43% of gynecology-obstetrics-pediatrics doctors had been reviewed, which was higher than the review rate of obstetricians in the United States of 33% [
Doctors from the Beijing or Shanghai areas were more likely to be reviewed and were also likely to receive more reviews than doctors from other areas in China. This might be because, first, Beijing and Shanghai have more famous hospitals than other parts of China [
Doctors from Level 3 hospitals were more likely to be reviewed, and were likely to receive more reviews than doctors from the Level 1 hospitals. Level 3 hospitals usually have more beds, better equipment, more highly skilled doctors, and deal with more challenging diseases. Again, this might suggest that patients with more serious health care problems and probably a longer interaction time are more likely to review their doctors. Interestingly, doctors from Level 2 hospitals were more likely to be reviewed than doctors from Level 1 hospitals, but received fewer reviews quantitatively compared to Level 1 hospitals. Specific reasons for this phenomenon needs research and data. But one thing that we should realize is that the Good Doctor website intentionally started their data collection from Level 3 hospitals or famous hospitals from large metropolitan areas in order to help consumers find good specialists. This strategy may have resulted in sample selection problems because on the Good Doctor website, 54% of the doctors were from Level 3 hospitals and 38% of the doctors were from Level 2 hospitals. Compared to the national data, where 49% of doctors work in Level 3 hospitals and 51% of doctors work in Level 2 hospitals (there are no Level 1 hospitals in the national aggregated data), the aggregated compositions are different. In other words, the national statistics indicate that the number of doctors in Level 3 and Level 2 hospitals are close in quantity but the Good Doctor website collected more doctors’ information from Level 3 hospitals to post online for patients to review.
Although, in total, there were more than 700,000 reviews in either the quantitative or the qualitative review sets, the average number of reviews per rated doctor was about 6.2, compared to that of the United States, which is 3 [
There might be a couple of possible reasons why Chinese consumers would like to review their doctors online. First, without a mature primary care system, Chinese consumers rely more on online doctor reviews to search for a doctor than their Western counterparts. Second, as a developing country, China usually has no formal pen-and-paper-based postvisit surveys to let patients review their health care providers. Some hospitals or clinics may provide a pen-and-paper-based “comment book” in the hospital lobby for patients to leave comments. But this is very informal and most patients ignore the comment book because most hospital lobbies are busy and crowded. Therefore, online reviewing may be the only way, or may be the first time, a Chinese patient can feel free to comment on their doctors with a structured measure. Also, the Good Doctor website was the only online doctor review website in China for a number of years, which may have allowed the Good Doctor website to accumulate more data. Again, further evidence and research are needed to answer the questions of why Chinese patients review their doctors online and how accurate the reviews are.
There has been no study about how Chinese consumers use or look at online doctor reviews or online health care information, and what factors may affect Chinese patients to participate in online doctor reviews. Some research has shown that 59% of American adults used the Internet for health care information and 16% of American adult Internet users have consulted online doctor reviews [
Many studies have found that most online doctor reviews are very positive [
In summary, many Chinese consumers have reviewed their doctors online as their Western counterparts have done. By April 11, 2014, 314,624 doctors from almost every medical specialty in China were listed on the Good Doctor website for Chinese consumers to review. There were 731,316 records of quantitative review, including both treatment effects and bedside manner, and 772,979 records of narrative textual review on 117,624 doctors from nine major specialty areas and many small unlisted specialty areas. The first contribution of this study is that it is the first, or one of the first, studies to examine the current status of online doctor reviews in China. Second, empirically, this study shows that like other countries, online doctor reviews in China covered almost all major medical specialties. Gynecology-obstetrics-pediatrics, surgery, and traditional Chinese medicine were more likely to be reviewed than the combined uncommon specialty areas, and gynecology-obstetrics-pediatrics and traditional Chinese medicine received more reviews than the combined specialty areas. But another major specialty area, internal medicine, was less likely to be reviewed than the combined specialty areas. All of the model estimates, except for surgery for the quantitative reviews, were statistically significant at the 5% level. Third, again like other countries, the majority of online doctor reviews were positive on the Good Doctor website. And finally, this study shows that the number of doctors may reach a stable level on the Good Doctor website and the number of reviews has been increasing.
This research has limitations. First, all the data used for analysis were from one single website, the Good Doctor, although this website is the largest and the first online doctor reviewing website in China. The website’s design change, database change, and strategy change may affect consumers’ decisions to post a review or not, or to post a positive or negative review. Second, the online doctor reviews were anonymous and there was no way to verify the truthfulness, hence, some of the reviews could have possibly been manipulated with some intention. However, the Good Doctor website does have a policy to check the reliability of reviews by asking the reviewers to leave a phone number on the website, which is not available to the public but only to the webmaster, in order to do random callbacks to verify the truthfulness of the reviews. Third, many doctors received a very limited number of reviews, on average 6 reviews per rated doctor, and those small numbers of reviews may not reflect the reality, or may only partially reflect the reality, of the doctors’ patient populations. Fourth, although China has the largest Internet population in the world, Internet accessibility is still low compared to developed countries around the world—45.8% of the Chinese population has Internet accessibility versus 84% in Germany, 84.2% in the United States, and 89.8% in the United Kingdom [
application programming interface
gynecology-obstetrics-pedicatrics
infectious disease
internal medicine
obstetrics
orthopedics
odds ratio
traditional Chinese medicine
I wish to give great thanks for the longstanding support from Mr Hang Wang, the founder of the Good Doctor website, and his technical team for this research.
None declared.