Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 07.08.19 in Vol 21, No 8 (2019): August

Preprints (earlier versions) of this paper are available at, first published May 07, 2019.

This paper is in the following e-collection/theme issue:


    What Do Patients Complain About Online: A Systematic Review and Taxonomy Framework Based on Patient Centeredness

    1School of Medicine and Health Management, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China

    2Library, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China

    3College of Engineering, Design and Physical Sciences, Brunel University London, London, United Kingdom

    4Department of Information Systems and Business Analytics, College of Business, Florida International University, Miami, FL, United States

    Corresponding Author:

    Jingdong Ma, MD, PhD

    School of Medicine and Health Management

    Tongji Medical College

    Huazhong University of Science and Technology

    No 13 Hangkong Road

    Qiaokou District



    Phone: 86 27 83692826



    Background: Complaints made online by patients about their health care experiences are becoming prevalent because of widespread worldwide internet connectivity. An a priori framework, based on patient centeredness, may be useful in identifying the types of issues patients complain about online across multiple settings. It may also assist in examining whether the determinants of patient-centered care (PCC) mirror the determinants of patient experiences.

    Objective: The objective of our study was to develop a taxonomy framework for patient complaints online based on patient centeredness and to examine whether the determinants of PCC mirror the determinants of patient experiences.

    Methods: First, the best fit framework synthesis technique was applied to develop the proposed a priori framework. Second, electronic databases, including Web of Science, Scopus, and PubMed, were searched for articles published between 2000 and June 2018. Studies were only included if they collected primary quantitative data on patients’ online complaints. Third, a deductive and inductive thematic analysis approach was adopted to code the themes of recognized complaints into the framework.

    Results: In total, 17 studies from 5 countries were included in this study. Patient complaint online taxonomies and theme terms varied. According to our framework, patients expressed most dissatisfaction with patient-centered processes (101,586/204,363, 49.71%), followed by prerequisites (appropriate skills and knowledge of physicians; 50,563, 24.74%) and the care environment (48,563/204,363, 23.76%). The least dissatisfied theme was expected outcomes (3651/204,363, 1.79%). People expressed little dissatisfaction with expanded PCC dimensions, such as involvement of family and friends (591/204,363, 0.29%). Variation in the concerns across different countries’ patients were also observed.

    Conclusions: Online complaints made by patients are of major value to health care providers, regulatory bodies, and patients themselves. Our PCC framework can be applied to analyze them under a wide range of conditions, treatments, and countries. This review has shown significant heterogeneity of patients’ online complaints across different countries.

    J Med Internet Res 2019;21(8):e14634




    As internet availability and usage grows worldwide, patients are spontaneously rating their experiences with physicians and hospitals by sharing their opinions about health encounters on the World Wide Web via mediums such as social media websites, Web-based consumer opinion platforms, and physician rating websites (PRWs) [1-5]. Previous research has demonstrated that patients are often influenced by peer-submitted comments posted on opinion and rating websites when making health care decisions [6,7]. On the basis of this notion, medical providers are able to leverage the information posted on such platforms to better comprehend patient experiences and engagement levels [4] and increase the understanding of patient frustrations and joy points during hospital visits [8-16]. By capturing patient data in real time, health care providers can use them as a quality metric to highlight insufficient physician performances or irregular events [5,17]. On the basis of extensive circumstantial evidence [18-20], countries such as the United Kingdom systematically collect data relating to patient experiences from their quality-reporting website (National Health Service, NHS choices) to support the further development of patient-centered care (PCC) [8,21-23].

    Given the intrinsic value of comments posted online by patients, it is important that health care providers make efficient use of the information collected. Practices observed from the adverse event taxonomy proposed by Harrison et al [24], which facilitated the collection and aggregation of data to compare findings, identify priorities, and develop wide-reaching patient safety solutions, demonstrated the momentousness of a unified, agreed framework with standardized concepts and terms. Despite the large volume of work published in this domain (eg, the studies by Reader et al [25] and Li et al [26]), currently available taxonomies for analyzing online complaints made by patients often lack standardized themes, terminology, and underlying unifying theory, creating difficulties in making sense of data that cannot be used to compare against other services, organizations, or countries. An operational and rigorous framework that classifies complaints made by patients, containing standardized concepts with agreed definitions and preferred terminology and establishing the relationships between concepts based on an explicit and nonoverlapping domain ontology, is required [24].

    When we take into consideration the well-developed PCC framework, which forms the basis for patient experience measurement systems in the United States, the United Kingdom, and other parts of Europe [27,28], we can see that it includes clear and proven terminologies with standardized dimensions and concepts. To create a patient-centric health care system that meets the needs and preferences of patients is one of the primary goals of numerous countries [17,21,27,29-32]. Therefore, it is feasible that we use the principles of PCC to guide the analysis of online complaints and examine whether the determinants of PCC are the same as the patient experience. To confirm this approach, a literature search was completed using a combination of keywords and subject headings, based on the defined concepts of patient complaints and PCC. Through analysis of the search results, it was identified that no study is yet to be completed that categorizes issues based on patient centeredness and that a generic taxonomy is required that appropriately analyzes issues against a wide range of conditions and in the context of different health care settings. These findings led to the following research questions (RQs) being posed:

    • RQ1: Have previous studies formed or adopted a credible taxonomy framework?
    • RQ2: Are available frameworks based on patient centeredness commensurate with what patients currently complain about online?
    • RQ3: Which dimensions of PCC constitute the focus of online complaints made by patients?
    • RQ4: Could a taxonomy framework allow us to identify the differences in patient complaints in a multicountry context?

    To answer these questions, a systematic review approach was used. First, we followed a process of synthesis [29] to propose our a priori framework. Then, comprehensive searches were conducted to systematically identify qualified studies relating to patient complaints online; at this point, data were extracted to match with the a priori framework; those that matched were compared between countries.

    To create the proposed framework, we synthesized all studies relating to PCC using the best fit framework synthesis technique proposed by Booth and Carroll [29]. First, we identified all the relevant frameworks or conceptual models that related to PCC, which are published in academic literature. At the forefront is the widely understood Picker Principles of Care framework, an internationally renowned approach used for measuring quality improvement in health care in the United States and the United Kingdom. Designed by the UK-based Institute for Healthcare Improvement, the PCC framework includes 8 dimensions: (1) respect for patient values, preferences, and expressed needs; (2) coordination and integration of care; (3) information and education; (4) physical comfort; (5) emotional support and alleviation of fear and anxiety; (6) involvement of family and friends; (7) continuity and transition; and (8) access to care. Although well adopted, the framework is considered a single-layer structure, which may lead to inefficiency in identifying homogeneous underlying problems. Brendan McCormack et al [33] developed a patient-centered framework comprising 4 constructs—prerequisites, the care environment, patient-centered processes, and expected outcomes—which was derived from Donabedian’s [34] structure-process-outcome assessment model. The proposed framework has been rigorously developed and tested in acute hospital settings [33] and is comprehensive enough to incorporate PCC dimensions. Second, we identified all relevant publications relating to the dimensions of PCC. Aside from the frameworks mentioned above, Kitson et al [35] and Rathert et al [36] completed systematic reviews of the PCC field and synthesized the common core elements of PCC. These 2 studies are highly cited and have been validated by a variety of follow-up studies. We compared and synthesized the dimensions of these 5 models. Third, we conducted a framework synthesis using thematic analysis [37]. Finally, grounded in the above, we developed an a priori framework with 4 domains, 8 categories, and 25 subcategories of online health care complaints based on patient centeredness, as shown in Figure 1.

    In the first layer of the framework, the prerequisites focus on health care professionals having appropriate skills and knowledge and the team of professionals being cohesive and cooperative. The care environment refers to the context in which care is delivered and includes supportive organizational systems and accessibility, in terms of geography, financial affordability, and availability. Patient-centered processes focus on delivering care through a range of activities that operationalize person-centered care. Expected outcomes relate to the results expected from effective PCC, addressing a patients’ physical and emotional needs.

    Figure 1. Proposed an a priori framework of online complaints based on patient centeredness.
    View this figure


    The systematic review reported on hereafter was conducted according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines[38]. As Reader et al [25] have previously systematically reviewed patient complaints, this study adopted some of their reporting items.

    Search Strategy

    The electronic databases of ISI Web of Science, Scopus, and PubMed were searched for articles published between 2000 (because of the explosive growth of internet usage around 2000 [30]) and June 2018. A medical librarian (SCH) developed a Boolean search strategy. Then, a doctoral student (JL) carried out the search strategy, which was revised, if required, by SCH. The keywords searched in the Title or Abstract fields related to (1) complaints (eg, comments OR ratings OR suggestions OR reviews OR feedback) and (2) online (eg, free text OR social media OR e-health OR virtual OR internet OR Facebook OR twitter), which were subject to inquiry (see Multimedia Appendix 1).

    Inclusion Criteria

    Studies were considered eligible if they were (a) related to the collection of primary quantitative data about patient complaints; (b) submitted by patients or third parties on their behalf; (c) comments uploaded to PRWs, organization’s websites, or any other online channel; or (d) conveyed in English to facilitate cross-country comparison.

    Exclusion Criteria

    Studies considered ineligible from this research included those that only referred to (a) physician ratings, (b) satisfaction questionnaires, (c) complaints made to non–health care–related organizations, or (d) qualitative analysis without quantifiable themes.

    Study Selection

    After removal of duplicates, JL screened the titles and abstracts of all the remaining records for relevance. In the next stage, the full-article text of the retrieved results was independently examined by JL and SCH for inclusion. Discrepancies were adjudicated by a senior researcher (JDM). A total of 17 papers were included in this systematic analysis, as illustrated in Figure 2.

    Figure 2. Preferred Reporting Items for Systematic reviews and Meta-Analyses flow chart.
    View this figure

    Data Extraction and Analysis

    Data were initially extracted by JL and subsequently checked by SCH, JL, and JDM, who together carried out the coding phase. The process consisted of the following phases: (1) descriptive and methodological data included in the study were extracted based on the items listed in the People-Centered and Integrated Health Services [33], (2) the number and categories of patient complaints were extracted, and (3) all themes or categories in selected studies were traversed and classified to match the initial categories. Data was analyzed using a thematic analysis approach adapted from the procedure outlined by Braun et al [37]. JL and SCH co-coded the issues of 2 papers to ensure that the coding framework and themes were commonly understood by the research team. JL and SCH independently coded the issues of the remaining 15 papers. In this paper, we achieved high interrater reliability (kappa=.82). Throughout the coding stage, to ensure consistency in categorizing the issue of complaints, 3 researchers (JL, SCH, and JDM) discussed to eliminate divergence.


    Search Results

    During the process of synthesizing identified literature, 35 studies were excluded for various reasons, including inability to distinguish between positive and negative comments [2,39], fuzzy quantity [11,16,41], channels of complaints [40,42], or only the number of high-frequency words of complaints mentioned [15,43]. Ultimately, 17 publications were identified as eligible. A wide range of data sources were represented, including (1) PRWs (10/17, 59%), such as RateMDs and China’s Good Doctor website; (2) government-managed health websites, such as NHS choices (2/17, 12%); (3) social networking sites (4/17, 24%); and (4) national online surveys (1/17, 6%). Of the 4 articles referring to social networking sites, 2 included data captured from tweets, whereas one was from Google+ reviews and one from Facebook.

    Descriptive and Methodological Data

    Through our analysis, we found that pertinent articles have emerged since 2012, with a steady increase observed ever since. Most of the research reported in the analyzed studies focused on PRWs or tweets examining country contexts, such as the United States, the United Kingdom, Germany, China, and Canada. From the data collected, it was identified that 10 articles focused on the United States (59%); 4 on the United Kingdom (24%); and 1 article each on China, Germany, and Canada. Most studies did not screen the departments in which the complainant arose (13/17, 76%), whereas 4 articles (24%) paid attention to complaints received from those dealing with a specific illness or encountering specific medical services. The number of complaints reported (or listed in the thematic analysis) in each study varied widely (average 6543, SD 15,547, range 36-57,028, and median 480).

    It was noted that the 17 articles included in our sample had different classification criteria, theme terms, and granularity of complaints. Among them, 6 articles (35%) classified complaints based on published categorization schemes, whereas 9 articles (53%) generated their own coding framework from scratch. The coding framework of 2 articles (12%) came from data source organizations. For example, Zhang et al [9] classified complaints according to the stages of medical consultation, which resulted in a 3-layer classification of stages, including medical consultation, diagnosis and treatment processes, and specific complaint attributions. Emmert et al [44] classified complaints according to the object being complained about and generated a 2-layer taxonomy, referring to both the object and specific complaint attributions. We assessed the quality of the included studies against interrater reliability/performance measures; number of layers; number of codes used, and if there were any definitions or descriptions; and examples provided, as shown in Table 1. The results indicated that the quality of studies varied: 8 did not report any classification measurement results and 4 studies used less than 6 codes, overall, whereas 15 did not provide definitions. A total of 13 studies did not provide examples of each theme/class. Thus, the taxonomy for patients’ online complaints is unstandardized, and it is deemed difficult to identify consistent problems arising in patient care.

    Table 1. A breakdown of descriptive and methodological data.
    View this table

    All papers specified the coders of the complaints; among them, 71% (n=12) were coded by the authors, 12% (n=2) were coded by data curators or investigators, and 18% (n=3) were coded automatically via machine learning techniques. In addition, 9 articles reported intercoder reliability (6 with the Cohen kappa coefficient, 2 with the Spearman correlation or Cronbach alpha, and 1 unspecified).

    Coding Results

    In total, across the 17 papers, 326 issue codes were used to code 154,762 complaints. Among them, 36 issue codes incorporating 9602 complaints were not classified into our classification framework for their ambiguous meaning of the category (such as others). When classifying all complaint codes, identified from the literature, into our complaint classification code system, we determined that a single code may include a number of new complaint codes that have been assigned individual type codes. The coding results containing concept explanations and issue numbers are provided in Table 2. Patients’ online complaints were seen to fit into the 4 domains, proposed in the a priori framework: (1) prerequisites, (2) patient-centered processes, (3) care environment, and (4) expected outcomes. From the complaints analyzed, it was concluded that patients have the most dissatisfaction with the patient-centered processes (101,586/204,363, 49.71%), followed by prerequisites (50,563/204,363, 24.74%), and care environment (48,563/204,363, 23.76%), with the least satisfied being expected outcomes (3651/204,363, 1.79%).

    The prerequisites domain referred to dissatisfaction with the professional skills and knowledge of the health care provider (48251/204,363, 23.61%) and cooperation between professionals in the medical team (2312/204,363, 1.13%). Among the 4 subcategories, the most common referred to comments about attributes of the patient-centered professional (38,314/204,363, 18.75%), which represents the explicit patient-centered personality traits of professionals.

    The domain with the most online complaints, patient-centered processes, was represented by a number of categories, as shown in Table 2. Within this domain, the greatest number of complaints related to a lack of open communication of knowledge, personal expertise, and clinical expertise between the patient and the professional (47,385/204,363, 23.19%). The second category care plan based on patient's individual needs collected 40,722 issues (40,722/204,363, 19.93%). The remaining 3 categories contained a small number of complaints, for example, patient participation as a respected and autonomous individual was represented by 4.00% (8186/204,363) of total issues. Among the 14 subcategories in this domain, information, communication, and education accounted for the majority of complaints, representing 22.80% (46,596/204,363) of total issues reported, whereas no complaints were reported on patient autonomy. The most frequently mentioned subtheme in the care environment domain was availability (28,784/204,363, 14.08%), which represented the timeliness of service and the accessibility of medical staff, facilities, and materials. Common problems mentioned in several articles were lengthy telephone calls made by the physician during consultation and difficulties in patients booking an appointment or seeing a clinician. It is worth noting that issues of therapeutic environment emerged in 12 articles. The expected outcomes domain contained complaints relating to physical comfort and physical care (3009/204,363, 1.47%), and emotional support for alleviation of anxiety issues (642/204,363, 0.31%).

    Table 2. Main results of the coding.
    View this table

    Country-Specific Differences in Complaints

    We analyzed online complaints from patients based in the United Kingdom, China, the United States, Germany, and Canada because of these being the focus of the studies identified in the literature. The distribution of the 4 domains of online health care complaints across different countries is displayed in Figure 3.

    With regard to care aspects that were complained about most frequently, Canadian (33/64, 52%) and British patients expressed the greatest dissatisfaction with prerequisites (34,828/69,746/, 49.94%), as compared with those of other nationalities. Referring to Canadian patient experiences of dental practice, dentists’ professional competence caused the greatest dissatisfaction (20/64, 31%), whereas the grievances of British patients to prerequisites were largely related to patients’ discontent over the attributes of the patient-centered professionals, which contributed to 42.74% (29,810 /69,746) of all complaint issues reviewed from British patients. Patients based in the United States attached significant importance to patient-centered processes (76,349/97,937, 77.96%), especially relating to information, communication, and education (41,133/97,937, 42.00%). German patients expressed negative comments to systemic problems (14,337/31,095), accounting for 46.11% of all complaints identified from German patients. In particular, poor therapeutic environments led to the most-complained-about topic. Chinese patients’ complaints relating to expected outcomes represented 9.82% (542/5521) of the total complaints made, far exceeding the sample average (3651/204,363, 1.79%).

    Figure 3. Distribution of 4 domains of online health care complaints across different countries.
    View this figure

    In addition to the aforementioned observations, it was identified that UK patients showed special features. First, they expressed dissatisfaction with the dimensions of care that patients in other countries did not complain about. Such dimensions included family and friends supported as caregivers, needs are anticipated, and transition and continuity of care. Second, UK patients expressed far-less-than-average complaints about certain dimensions of care than those in other countries, such as financial accessibility (0% vs 1.45%) and information, communication, and education (1.94% vs 22.80%). The proportion of complaints from Chinese patients to experts’ professional competence and physical comfort was significantly higher than that of patients from other countries.


    Principal Findings

    The results of this systematic review show that there is not yet a widely adopted taxonomy framework for classifying patient complaints made online in various settings. This means that studies have defined and classified complaints differently, resulting in limited comparability between available studies and research contexts [24,25,57]. To eliminate this gap, we have developed an a priori framework of PCC, which can incorporate patient complaints online. The NHS Institute for Innovation and Improvement found that PCC frameworks (eg, Picker framework) are broadly appropriate for “what matters most” to patients [58]. We have further validated the scope of the PCC frameworks using data from patient complaints online. Our research is considered beneficial for identifying gaps in the evidence base for patient experience expressed online, which has been identified as domains of PCC.

    From the analysis of investigated studies, we have identified that the generic themes of prerequisites, the care environment, and the processes were complained about the most or, put it another way, greatly valued by the majority of patients, regardless of where they came from, and should be deemed priorities for PCC. To be more specific, a practicing clinician should be trained more on these dimensions: attributes of patient-centered professional, information, communication, and education.

    Although involvement of family and friends is increasingly viewed as an important component of PCC [1,59-61], patients seldom expressed dissatisfaction with it in our research; apparently far less complained about the low participation of themselves. Compared with technical competence, which constituted a fundamental aspect of health care provision [23], interpersonal attributes of professionals were much more likely to be evaluated by patients; the potential implications of this are twofold: first, consistent with the study by Jia Li et al [26], patients’ needs have different hierarchy, which were firstly stated by Maslow [62], and second, patients have uninformed expectation—“patients are not capable or are reluctant to communicate their expectations” [61], which was validated by Rothenfluh and Schulz [63]. Although there were common concerns across different countries’ patients, variation existed in this study as well. Overall, 5 countries included in our study have different health care systems in terms of health care insurance, drug pricing, physician compensation, etc, and lead to disparities in quality of care, care coordination, and physician education: for example, health care in countries such as the United Kingdom and Canada is publicly financed and the coverage is universal; however, the health coverage of America remains fragmented, with numerous private and public sources, as well as wide gaps in insured rates across the US population [64].

    In general, UK patients contributed most of the complaints about the attributes of patient-centered professionals. This finding correlates with that of the NHS Institute for Innovation and Improvement [58]. Besides, British patients had a higher pursuit as they voiced high-level needs such as involvement of family and friends (591 issues), anticipated needs (462 issues), feedback mechanisms to measure patient experience (37 issues), and emotional support (642 issues). British patients were also the only nationality to convey discontent over these aspects of care. Given that the NHS has been collecting data on patients’ experience of care for over 10 years [58] and professionals are occasionally accused of being incompetent in satisfying patients’ needs to be treated as a person, eliminating the gap between knowing and doing is crucial. From another perspective, UK patients are assumed to be more informed about PCC by virtue of a variety of regular national health and social care surveys carried out in the United Kingdom, with frameworks of several surveys adopting Picker’s PCC principles being available [58,65]; however, this hypothesis awaits further confirmation.

    Despite nearly none of the British patients in this study expressing discontent over financial accessibility (0/69,746, 0.00%) and physical outcomes (376/69,746, 0.54%), approximately 10% (542/5521, 9.82%; 441/5521, 7.99%) of Chinese patients’ complaints referred to these issues. Although the coverage by publicly financed health insurance in China is near-universal, out-of-pocket spending per capita represented approximately 32% of total health expenditures in 2014 [64]. Aside from this, high registration fees, the formidable markup by ticket touts operating in health care locations [9], fees related to excessive tests and treatments [26,66,67], and insurance reimbursement obstacles (cross-regional medical treatment) [67] constitute Chinese patients’ financial barriers. It is likely that physician-dominated decision-making [68], inadequate communication, and patient distrust of doctors [66,68,69] have led to dissatisfaction with their physical outcomes.

    Looking into the aspects of care that deeply concerned German patients, we find it necessary to improve the therapeutic environment of health care provision, in terms of privacy and entertainment, as well as maintaining a continuous healing relationship through telecommunication and house visits [44]. Given that studies on patient experience in Germany have not taken these vulnerable care aspects into account [70-72], these dissatisfaction factors should be tested in future research.

    In this study, patients based in the United States conveyed great dissatisfaction for information, communication, and education (41,133/97,937, 42.00%), as well as coordination and integration of care (33,520/97,937, 34.23%). Previous work on patient-centered communication demonstrated a positive correlation between skilled physician communication and patient satisfaction [73-75]. Utilizing patient-centered communication guidelines and codes of conduct, such as physicians’ humility and communication training for physicians and medical students, may bring better patient experience and diminish patient complaints [70,73-76]. As for coordination and integration of care, as necessitated when patients encounter long waiting times in hospitals or disorganized operations, previous studies have focused on hospital-level care coordination strategies associated with better patient experience [77]. Besides, it is envisaged that information technologies can reduce the need to craft laborious, case-by-case strategies for coordinating patient care [30,78,79].

    Our findings suggest that it is feasible to identify gaps in evidence bases for patient experiences, which have been identified as domains of PCC. It was observed that differences and commonalities coexist across countries, after applying the proposed taxonomy framework, and we found that there is much leeway for the countries of interest to seek improvement in patient-centeredness.


    Patient complaints online can indicate weaknesses in the health care system through the eyes of the patients’ themselves. The proposed PCC framework can be applied to analyze the complaints under a wide PCC range of conditions, treatments, and countries. This review has shown significant heterogeneity of patients’ online complaints in different countries, attributable to the diversity in culture, health care institution, and health literacy. Further work is required to apply the framework, using a plethora of data sources, to compare with other services, organizations, and countries or within the health care service over time, that is, a longitudinal study.

    All RQs, proposed in the Introduction of this paper, were answered through conduction of the systematic review. Despite certain studies classifying patient complaints online, none were found to include or adopt a credible taxonomy framework. The proposed PCC framework aligns with what patients currently complain about online. By applying the taxonomy, results show that health professionals’ skills and knowledge, open communication of knowledge, and system issues of PCC constitute the focus of online complaints made by patients. In addition, the differences in patient complaints in a multicountry context are discussed.


    As always, there are several limitations to this study. First, it was based on searches in merely 3 databases and focused only on currently available peer-reviewed literature; for this reason, we may have missed information in the gray literature. Second, regarding the small number of included articles, because of artificial screening, and the resulting relatively small sample size, our conclusions, especially the country-specific ones, may not be free of overgeneralizations and missing targets, even by taking into account the “community of common destiny.” Finally, our interpretation of the concepts and scope of the various categories of complaints included in the article may not be fully consistent with the authors of the included papers, especially if the explanations or quotes were not given. Considering the limits to the time, space and researcher resources of this study, it is, nonetheless, a worthy trial that merits further exploration.


    This work was supported by Fundamental Research Funds for the Central Universities, HUST: 2015AE017.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Search strategy.

    PDF File (Adobe PDF File), 57KB


    1. Drevs F, Hinz V. Who chooses, who uses, who rates: the impact of agency on electronic word-of-mouth about hospitals stays. Health Care Manage Rev 2014;39(3):223-233. [CrossRef] [Medline]
    2. Griffiths A, Leaver MP. Wisdom of patients: predicting the quality of care using aggregated patient feedback. BMJ Qual Saf 2018 Dec;27(2):110-118 [FREE Full text] [CrossRef] [Medline]
    3. Hopkins B, Yamaguchi JT, Cloney MB, Shlobin NA, Dahdaleh NS. Effects of the physician payments sunshine act on the patient experience and perception of care amongst neurosurgeons: a comparative study of online PRW ratings and industry payments. Clin Neurol Neurosurg 2019 Jan;176:127-132. [CrossRef] [Medline]
    4. Rozenblum R, Greaves F, Bates DW. The role of social media around patient experience and engagement. BMJ Qual Saf 2017 Dec;26(10):845-848. [CrossRef] [Medline]
    5. Greaves F, Ramirez-Cano D, Millett C, Darzi A, Donaldson L. Harnessing the cloud of patient experience: using social media to detect poor quality healthcare. BMJ Qual Saf 2013 Mar;22(3):251-255. [CrossRef] [Medline]
    6. Burkle CM, Keegan MT. Popularity of internet physician rating sites and their apparent influence on patients' choices of physicians. BMC Health Serv Res 2015 Sep 26;15:416 [FREE Full text] [CrossRef] [Medline]
    7. Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. Parental awareness and use of online physician rating sites. Pediatrics 2014 Oct;134(4):e966-e975 [FREE Full text] [CrossRef] [Medline]
    8. Lagu T, Goff SL, Hannon NS, Shatz A, Lindenauer PK. A mixed-methods analysis of patient reviews of hospital care in England: implications for public reporting of health care quality data in the United States. Jt Comm J Qual Patient Saf 2013 Jan;39(1):7-15. [CrossRef] [Medline]
    9. Zhang W, Deng Z, Hong Z, Evans R, Ma J, Zhang H. Unhappy patients are not alike: content analysis of the negative comments from China's good doctor website. J Med Internet Res 2018 Dec 25;20(1):e35 [FREE Full text] [CrossRef] [Medline]
    10. Lagu T, Hannon NS, Rothberg MB, Lindenauer PK. Patients' evaluations of health care providers in the era of social networking: an analysis of physician-rating websites. J Gen Intern Med 2010 Sep;25(9):942-946 [FREE Full text] [CrossRef] [Medline]
    11. Asanad K, Parameshwar PS, Houman J, Spiegel BM, Daskivich TJ, Anger JT. Online physician reviews in female pelvic medicine and reconstructive surgery: what do patients really want? Female Pelvic Med Reconstr Surg 2018;24(2):109-114. [CrossRef] [Medline]
    12. Bardach NS, Lyndon A, Asteria-Peñaloza R, Goldman LE, Lin GA, Dudley RA. From the closest observers of patient care: a thematic analysis of online narrative reviews of hospitals. BMJ Qual Saf 2016 Dec;25(11):889-897 [FREE Full text] [CrossRef] [Medline]
    13. Wallace BC, Paul MJ, Sarkar U, Trikalinos TA, Dredze M. A large-scale quantitative analysis of latent factors and sentiment in online doctor reviews. J Am Med Inform Assoc 2014;21(6):1098-1103 [FREE Full text] [CrossRef] [Medline]
    14. Rastegar-Mojarad M, Ye Z, Wall D, Murali N, Lin S. Collecting and analyzing patient experiences of health care from social media. JMIR Res Protoc 2015 Jul 2;4(3):e78 [FREE Full text] [CrossRef] [Medline]
    15. Greaves F, Ramirez-Cano D, Millett C, Darzi A, Donaldson L. Use of sentiment analysis for capturing patient experience from free-text comments posted online. J Med Internet Res 2013 Nov 1;15(11):e239 [FREE Full text] [CrossRef] [Medline]
    16. Noll D, Mahon B, Shroff B, Carrico C, Lindauer SJ. Twitter analysis of the orthodontic patient experience with braces vs invisalign. Angle Orthod 2017 May;87(3):377-383. [CrossRef] [Medline]
    17. Rozenblum R, Bates DW. Patient-centred healthcare, social media and the internet: the perfect storm? BMJ Qual Saf 2013 Mar;22(3):183-186. [CrossRef] [Medline]
    18. Manary MP, Boulding W, Staelin R, Glickman SW. The patient experience and health outcomes. N Engl J Med 2013 Jan 17;368(3):201-203. [CrossRef] [Medline]
    19. Aiken LH, Sermeus W, van den Heede K, Sloane DM, Busse R, McKee M, et al. Patient safety, satisfaction, and quality of hospital care: cross sectional surveys of nurses and patients in 12 countries in Europe and the United States. Br Med J 2012 Mar 20;344:e1717 [FREE Full text] [CrossRef] [Medline]
    20. GOV.UK. 2008. High Quality Care for All: NHS Next Stage Review Final Report   URL: https:/​/www.​​government/​publications/​high-quality-care-for-all-nhs-next-stage-review-final-report [accessed 2018-09-15]
    21. Greaves F, Millett C, Nuki P. England's experience incorporating 'anecdotal' reports from consumers into their national reporting system: lessons for the United States of what to do or not to do? Med Care Res Rev 2014 Oct;71(5 Suppl):65S-80S. [CrossRef] [Medline]
    22. Asprey A, Campbell JL, Newbould J, Cohn S, Carter M, Davey A, et al. Challenges to the credibility of patient feedback in primary healthcare settings: a qualitative study. Br J Gen Pract 2013 Mar;63(608):e200-e208 [FREE Full text] [CrossRef] [Medline]
    23. Brookes G, Baker P. What does patient feedback reveal about the NHS? A mixed methods study of comments posted to the NHS choices online service. BMJ Open 2017 Dec 27;7(4):e013821 [FREE Full text] [CrossRef] [Medline]
    24. Harrison R, Walton M, Healy J, Smith-Merry J, Hobbs C. Patient complaints about hospital services: applying a complaint taxonomy to analyse and respond to complaints. Int J Qual Health Care 2016 Apr;28(2):240-245. [CrossRef] [Medline]
    25. Reader TW, Gillespie A, Roberts J. Patient complaints in healthcare systems: a systematic review and coding taxonomy. BMJ Qual Saf 2014 Aug;23(8):678-689 [FREE Full text] [CrossRef] [Medline]
    26. Li J, Liu M, Li X, Liu X, Liu J. Developing embedded taxonomy and mining patients' interests from web-based physician reviews: mixed-methods approach. J Med Internet Res 2018 Dec 16;20(8):e254 [FREE Full text] [CrossRef] [Medline]
    27. Giuseppe P. Picker: Patient Centred Care Improvement. 2016. Person-centred care in Europe: a cross-country comparison of health system performance, strategies and structures   URL: https:/​/www.​​wp-content/​uploads/​2016/​02/​12-02-16-Policy-briefing-on-patient-centred-care-in-Europe.​pdf
    28. Redding D, Hutchinson S. National Voices. 2017. Person-Centred Care in 2017   URL: [accessed 2019-04-10]
    29. Booth A, Carroll C. How to build up the actionable knowledge base: the role of 'best fit' framework synthesis for studies of improvement in healthcare. BMJ Qual Saf 2015 Nov;24(11):700-708 [FREE Full text] [CrossRef] [Medline]
    30. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001.
    31. Ministry of Solidarity and Health. 2017. [The 2018-2022 National Health Strategy]   URL: https:/​/solidarites-sante.​​systeme-de-sante-et-medico-social/​strategie-nationale-de-sante/​article/​la-strategie-nationale-de-sante-2018-2022 [accessed 2018-11-18]
    32. NHS England. Creating the Conditions for Person-Centred Care   URL: [accessed 2018-11-18]
    33. McCormack B, McCance TV. Development of a framework for person-centred nursing. J Adv Nurs 2006 Dec;56(5):472-479. [CrossRef] [Medline]
    34. Donabedian A. The quality of care. How can it be assessed? J Am Med Assoc 1988;260(12):1743-1748. [CrossRef] [Medline]
    35. Kitson A, Marshall A, Bassett K, Zeitz K. What are the core elements of patient-centred care? A narrative review and synthesis of the literature from health policy, medicine and nursing. J Adv Nurs 2013 Jan;69(1):4-15. [CrossRef] [Medline]
    36. Rathert C, Wyrwich MD, Boren SA. Patient-centered care and outcomes: a systematic review of the literature. Med Care Res Rev 2013 Aug;70(4):351-379. [CrossRef] [Medline]
    37. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3(2):77-101. [CrossRef]
    38. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
    39. Hawkins JB, Brownstein JS, Tuli G, Runels T, Broecker K, Nsoesie EO, et al. Measuring patient-perceived quality of care in US hospitals using Twitter. BMJ Qual Saf 2016 Dec;25(6):404-413 [FREE Full text] [CrossRef] [Medline]
    40. Wagland R, Recio-Saucedo A, Simon M, Bracher M, Hunt K, Foster C, et al. Development and testing of a text-mining approach to analyse patients' comments on their experiences of colorectal cancer care. BMJ Qual Saf 2016 Dec;25(8):604-614. [CrossRef] [Medline]
    41. Kowalski R. Patients’ Written Reviews as a Resource for Public Healthcare Management in England. 2017 Presented at: EICN'17; September 18-20, 2017; Lund, Sweden p. 545-550. [CrossRef]
    42. Corner J, Wagland R, Glaser A, Richards SM. Qualitative analysis of patients' feedback from a PROMs survey of cancer patients in England. BMJ Open 2013 Apr 10;3(4):e002316 [FREE Full text] [CrossRef] [Medline]
    43. Paul MJ, Wallace BC, Dredze M. What Affects Patient (Dis)satisfaction? Analyzing Online Doctor Ratings With a Joint Topic-Sentiment Model. In: Expanding the Boundaries of Health Informatics Using AI. 2013 Presented at: AAAI'13; July 14-15, 2013; Bellevue, Washington.
    44. Emmert M, Meier F, Heider AK, Dürr C, Sander U. What do patients say about their physicians? An analysis of 3000 narrative comments posted on a German physician rating website. Health Policy 2014 Oct;118(1):66-73. [CrossRef] [Medline]
    45. Alemi F, Torii M, Clementz L, Aron DC. Feasibility of real-time satisfaction surveys through automated analysis of patients' unstructured comments and sentiments. Qual Manag Health Care 2012;21(1):9-19. [CrossRef] [Medline]
    46. López A, Detz A, Ratanawongsa N, Sarkar U. What patients say about their doctors online: a qualitative content analysis. J Gen Intern Med 2012 Jun;27(6):685-692 [FREE Full text] [CrossRef] [Medline]
    47. Detz A, López A, Sarkar U. Long-term doctor-patient relationships: patient perspective from online reviews. J Med Internet Res 2013 Jul;15(7):e131 [FREE Full text] [CrossRef] [Medline]
    48. Greaves F, Laverty AA, Cano DR, Moilanen K, Pulman S, Darzi A, et al. Tweets about hospital quality: a mixed methods study. BMJ Qual Saf 2014 Oct;23(10):838-846 [FREE Full text] [CrossRef] [Medline]
    49. Macdonald ME, Beaudin A, Pineda C. What do patients think about dental services in Quebec? Analysis of a dentist rating website. J Can Dent Assoc 2015;81:f3 [FREE Full text] [Medline]
    50. Lagu T, Goff SL, Craft B, Calcasola S, Benjamin EM, Priya A, et al. Can social media be used as a hospital quality improvement tool? J Hosp Med 2016 Jan;11(1):52-55 [FREE Full text] [CrossRef] [Medline]
    51. Trehan SK, DeFrancesco CJ, Nguyen JT, Charalel RA, Daluiski A. Online Patient Ratings of Hand Surgeons. J Hand Surg 2016;41:98-103. [CrossRef] [Medline]
    52. Cunningham M, Wells M. Qualitative analysis of 6961 free-text comments from the first National Cancer Patient Experience Survey in Scotland. BMJ Open 2017 Dec 15;7(6):e015726 [FREE Full text] [CrossRef] [Medline]
    53. James TL, Calderon ED, Cook DF. Exploring patient perceptions of healthcare service quality through analysis of unstructured feedback. Expert Syst Appl 2017 Apr;71:479-492. [CrossRef]
    54. Xu S, Atanelov Z, Bhatia AC. Online patient-reported reviews of Mohs micrographic surgery: qualitative analysis of positive and negative experiences. Cutis 2017 Feb;99(2):E25-E29. [Medline]
    55. King RH, Stanley J, Baum N. Hard internet truths: 34,748 online reviews reveal what patients really want from doctors. J Med Pract Manage 2016;31(5):309-312. [Medline]
    56. Emmert M, Meszmer N, Schlesinger M. A cross-sectional study assessing the association between online ratings and clinical quality of care measures for US hospitals: results from an observational study. BMC Health Serv Res 2018 Dec 5;18(1):82 [FREE Full text] [CrossRef] [Medline]
    57. Runciman W, Hibbert P, Thomson R, van der Schaaf T, Sherman H, Lewalle P. Towards an international classification for patient safety: key concepts and terms. Int J Qual Health Care 2009 Feb;21(1):18-26 [FREE Full text] [CrossRef] [Medline]
    58. Robert G, Cornwell J. Wales NHS. 2011 Nov. What Matters to Patients? Developing the Evidence Base for Measuring and Improving Patient Experience   URL: http:/​/www.​​sites3/​documents/​420/​Final%20Project%20Report%20pdf%20doc%20january%202012%20(2).​pdf [accessed 2018-10-09]
    59. National Center for Interprofessional Practice and Education. 2015. Picker Institute’s Eight Principles of Patient-Centered Care   URL: https:/​/nexusipe.​org/​informing/​resource-center/​picker-institute%E2%80%99s-eight-principles-patient-centered-care [accessed 2018-12-11]
    60. Gerteis M, Delbanco TL, Daley J, Edgman-Levitan S. Through the Patient's Eyes: Understanding and Promoting Patient-Centered Care. San Francisco, CA: Jossey-Bass; 1993.
    61. Sofaer S, Firminger K. Patient perceptions of the quality of health services. Annu Rev Public Health 2005;26:513-559. [CrossRef] [Medline]
    62. Maslow AH. A theory of human motivation. Psychol Rev 1943;50(4):370-396. [CrossRef]
    63. Rothenfluh F, Schulz PJ. Physician rating websites: what aspects are important to identify a good doctor, and are patients capable of assessing them? A mixed-methods approach including physicians' and health care consumers' perspectives. J Med Internet Res 2017 Dec 1;19(5):e127 [FREE Full text] [CrossRef] [Medline]
    64. International Health Care System Profiles. Country Profiles   URL: [accessed 2019-05-30]
    65. Coulter A, Locock L, Ziebland S, Calabrese J. Collecting data on patient experience is not enough: they must be used to improve care. Br Med J 2014 Mar 26;348:g2225. [CrossRef] [Medline]
    66. Wang X, Jiang R, Li J, Chen J, Burström B, Burström K. What do patients care most about in China's public hospitals? Interviews with patients in Jiangsu Province. BMC Health Serv Res 2018 Dec 8;18(1):97 [FREE Full text] [CrossRef] [Medline]
    67. Gong X. The health care gap between China and America. Ann Transl Med 2014 Apr;2(4):39 [FREE Full text] [CrossRef] [Medline]
    68. Ting X, Yong B, Yin L, Mi T. Patient perception and the barriers to practicing patient-centered communication: a survey and in-depth interview of Chinese patients and physicians. Patient Educ Couns 2016 Mar;99(3):364-369. [CrossRef] [Medline]
    69. Hung LM, Shi L, Wang H, Nie X, Meng Q. Chinese primary care providers and motivating factors on performance. Fam Pract 2013 Oct;30(5):576-586. [CrossRef] [Medline]
    70. Breuning M, Lucius-Hoene G, Burbaum C, Himmel W, Bengel J. [Patient experiences and patient centeredness: the website project DIPEx Germany]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2017 Apr;60(4):453-461. [CrossRef] [Medline]
    71. Kraska RA, Weigand M, Geraedts M. Associations between hospital characteristics and patient satisfaction in Germany. Health Expect 2017 Dec;20(4):593-600 [FREE Full text] [CrossRef] [Medline]
    72. Schoenfelder T, Klewer J, Kugler J. Determinants of patient satisfaction: a study among 39 hospitals in an in-patient setting in Germany. Int J Qual Health Care 2011 Oct;23(5):503-509. [CrossRef] [Medline]
    73. King A, Hoppe RB. 'Best practice' for patient-centered communication: a narrative review. J Grad Med Educ 2013 Sep;5(3):385-393 [FREE Full text] [CrossRef] [Medline]
    74. Brueck MK, Salib AM. Strategies for acing the fundamentals and mitigating legal and ethical consequences of poor physician-patient communication. AMA J Ethics 2017 Mar 1;19(3):289-295 [FREE Full text] [CrossRef] [Medline]
    75. Ruberton PM, Huynh HP, Miller TA, Kruse E, Chancellor J, Lyubomirsky S. The relationship between physician humility, physician-patient communication, and patient health. Patient Educ Couns 2016 Dec;99(7):1138-1145. [CrossRef] [Medline]
    76. Boissy A, Windover AK, Bokar D, Karafa M, Neuendorf K, Frankel RM, et al. Communication skills training for physicians improves patient satisfaction. J Gen Intern Med 2016 Dec;31(7):755-761 [FREE Full text] [CrossRef] [Medline]
    77. Figueroa JF, Feyman Y, Zhou X, Joynt Maddox K. Hospital-level care coordination strategies associated with better patient experience. BMJ Qual Saf 2018 Oct;27(10):844-851. [CrossRef] [Medline]
    78. McKay C, Vanaskie K. Partnering for success: the role of the nurse leader in health information technology implementation for coordination of care. Nurse Lead 2018 Dec;16(6):385-388. [CrossRef]
    79. Chang L, Wanner KJ, Kovalsky D, Smith KL, Rhodes KV. 'It's really overwhelming': patient perspectives on care coordination. J Am Board Fam Med 2018;31(5):682-690 [FREE Full text] [CrossRef] [Medline]


    NHS: National Health Service
    PCC: patient-centered care
    PRW: physician rating website
    RQ: research question

    Edited by G Eysenbach; submitted 07.05.19; peer-reviewed by S Lin, R Druz; comments to author 29.05.19; revised version received 05.06.19; accepted 09.06.19; published 07.08.19

    ©Jing Liu, Shengchao Hou, Richard Evans, Chenxi Xia, Weidong Xia, Jingdong Ma. Originally published in the Journal of Medical Internet Research (, 07.08.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.