Published on in Vol 16, No 2 (2014): February

Hospital Website Rankings in the United States: Expanding Benchmarks and Standards for Effective Consumer Engagement

Hospital Website Rankings in the United States: Expanding Benchmarks and Standards for Effective Consumer Engagement

Hospital Website Rankings in the United States: Expanding Benchmarks and Standards for Effective Consumer Engagement

Original Paper

1Departments of Family Medicine and Biomedical Informatics, College of Medicine, The Ohio State University, Columbus, OH, United States

2Department of Family Medicine, College of Medicine, The Ohio State University, Columbus, OH, United States

3Bryan School of Business and Economics, University of North Carolina Greensboro, Greensboro, NC, United States

4Department of Health Care Organization and Policy, University of Alabama at Birmingham, Birmingham, AL, United States

Corresponding Author:

Timothy R Huerta, MS, PhD

Departments of Family Medicine and Biomedical Informatics

College of Medicine, The Ohio State University

261 Northwood-High Building

2231 North High Street

Columbus, OH, 43201

United States

Phone: 1 614 366 6636

Fax:1 614 293 7215

Email: Timothy.Huerta@osumc.edu


Background: Passage of the Patient Protection and Affordable Care Act (ACA) increased the roles hospitals and health systems play in care delivery and led to a wave of consolidation of medical groups and hospitals. As such, the traditional patient interaction with an independent medical provider is becoming far less common, replaced by frequent interactions with integrated medical groups and health systems. It is thus increasingly important for these organizations to have an effective social media presence. Moreover, in the age of the informed consumer, patients desire a readily accessible, electronic interface to initiate contact, making a well-designed website and social media strategy critical features of the modern health care organization.

Objective: The purpose of this study was to assess the Web presence of hospitals and their health systems on five dimensions: accessibility, content, marketing, technology, and usability. In addition, an overall ranking was calculated to identify the top 100 hospital and health system websites.

Methods: A total of 2407 unique Web domains covering 2785 hospital facilities or their parent organizations were identified and matched against the 2009 American Hospital Association (AHA) Annual Survey. This is a four-fold improvement in prior research and represents what the authors believe to be a census assessment of the online presence of US hospitals and their health systems. Each of the five dimensions was investigated with an automated content analysis using a suite of tools. Scores on the dimensions are reported on a range from 0 to 10, with a higher score on any given dimension representing better comparative performance. Rankings on each dimension and an average ranking are provided for the top 100 hospitals.

Results: The mean score on the usability dimension, meant to rate overall website quality, was 5.16 (SD 1.43), with the highest score of 8 shared by only 5 hospitals. Mean scores on other dimensions were between 4.43 (SD 2.19) and 6.49 (SD 0.96). Based on these scores, rank order calculations for the top 100 websites are presented. Additionally, a link to raw data, including AHA ID, is provided to enable researchers and practitioners the ability to further explore relationships to other dynamics in health care.

Conclusions: This census assessment of US hospitals and their health systems provides a clear indication of the state of the sector. While stakeholder engagement is core to most discussions of the role that hospitals must play in relation to communities, management of an online presence has not been recognized as a core competency fundamental to care delivery. Yet, social media management and network engagement are skills that exist at the confluence of marketing and technical prowess. This paper presents performance guidelines evaluated against best-demonstrated practice or independent standards to facilitate improvement of the sector’s use of websites and social media.

J Med Internet Res 2014;16(2):e64

doi:10.2196/jmir.3054

Keywords



More than 80% of adults reported using Internet resources to support health care decisions in 2011 [1,2]. Thus, in many instances, a hospital’s home page is the first point of contact for consumers [3,4]. As a result, the website for a hospital or health system has become an important communication channel for marketing to current and potential customers, as well as to visitors accompanying a patient [5-7]. Customers’ evaluations of a hospital’s website, and by extension their perceptions of the facility itself, will be based in part on comparisons to their experiences using other consumer websites such as Amazon and eBay [8,9]. If a hospital’s website does not conform to or exceed a customer’s expectations based on their previous experiences, negative inferences about facility quality will influence the decision-making process [10].

Reliance on search engines has resulted in individuals taking varied routes to their website destinations. Weaver et al [11] found that people seeking illness-related information behave differently than those seeking wellness information. Based on these trends, many hospital or health system websites have begun to include tools and information for patients and visitors that make navigating complex health encounters more user-friendly and that create a positive organizational image [12]. In so doing, hospitals are increasingly seeking to take on the role of trusted adviser, a role that is closely aligned with the accountable care organization (ACO) model in which health care providers work to empower patients to improve population health [7,13,14].

The competitive advantage gained from building an effective Web presence has led researchers to establish accessibility, content, marketing, and technical standards that define best demonstrated practices in website design [15-17]. In addition, the Health Information Technology Institute has issued standards for health care websites, including credibility, content, disclosure, links, design, interactivity, and caveats [18]. As a result, an exploration of US hospital and affiliated health system websites against the design standards used in other commercial endeavors is warranted to establish the state of the field.

The purpose of this article is to identify the degree to which hospitals and their health system websites comply with Internet-industry standards for commercial usability [19]. Using an automated Web crawler, we evaluated four dimensions—accessibility, content, marketing, and technology—using weighted multi-item scales. In addition, we used a weighted composite overall score to measure each website’s quality across all four dimensions. The authors believe this analysis to be a census assessment for the online presence of hospitals and their health systems in the United States.

For hospital and health system decision makers, these analyses provide quantifiably objective, and immediately actionable, recommendations for enhancing the quality of their organization’s website. Compared to other health information technology (HIT) upgrades that are being made to meet meaningful use goals set by the federal government, the capital investments required to create a state-of-the-art website are relatively modest and immediately visible. Additionally, as outlined above, having an effective website can create a competitive advantage when attracting consumers.

There is a dearth of evaluation studies on hospital websites. A number of factors play into this issue. Much of the research on the role of the Web in health include issues of accuracy, understandability, provenance, and timeliness. Williams et al [20] discuss a typology offered by Trochim on the purpose of websites and note that while the organizations don’t keep their purposes neatly within the lines, they can still be assessed on the raison d’être—their reason for being and the services they aspire to provide. As such, there have been some efforts to study the content of health sites on a more specific basis—by judging the quality of the information as opposed to the technical merits of the system that is used for conveying that material. The American Public Health Association (APHA) offers a comprehensive statement on these issues [21].

First, while numerous studies exist focused on health information, hospitals are generally not seen as a portal for that purpose. This can be seen in the samples from which many of the health assessments are drawn—generally sites like WebMD, which seek to provide health information [20]—as opposed to local hospitals. While some facilities are large enough to have a national draw, and therefore serve the former purpose (eg, the Mayo Clinic, the Cleveland Clinic), most hospitals do not seek to provide health information, per se. Rather, they seek to provide information on services provided at the facility. As a result, hospitals serve a different role in the community, which generally is the delivery of health care services. This relationship is transforming as a result of the move to the ACO model in the United States, where hospitals are taking a greater responsibility for the care of the panels of patients they serve.

Ivory [22] notes that website evaluation is a moving target, suggesting that early assessments of evaluation focused on technical assessments of speed, while those have shifted over time. While some scholars have noted that human participation is fundamental to evaluating the usability of Web content, to ignore the value of automated assessment is to ignore the variability of experience provided to the end user by virtue of a site’s failure to adhere to standards. Chiang and Starren [23] detail an assessment of consumer health website accessibility by users with sensory and physical disabilities, noting that W3C compliance is a minimum standard for ensuring accessibility, and adherence to that standard protocol should be framed as foundational and a precursor to more detailed assessment.


Overview

The names, cities, and states for every “non-government, not-for-profit (NFP)” or “investor-owned, for-profit” general medical or surgical hospital listed in the AHA 2009 Annual Survey in the United States (N=3523) were used in both Google and Bing to identify the first three returns for the search engines. These links were inspected to identify whether a hospital matching the facility of interest could be identified. In cases where a matching facility could not be found, an additional manual search was conducted in Google to attempt to locate the facility. Web site domains were included if and only if they could be attributed to the facility or a parent organization that was also a health services delivery organization. For instance, several critical access hospitals were excluded because they maintained a Web presence under a county government’s Web domain. Website testing took place during the month of May 2013.

It should be noted that some organizations may have been a member of a health system, but also maintained their own domain. For instance, Northeast Arkansas Baptist Health System (neabaptist.com) is a hospital associated with Baptist Health System (baptistonline.org). In cases where a facility had its own domain, we assessed that domain separate and apart from the network or health system of which it was a part. The website of each organization was secured using a custom-built Web crawler. The Web crawler begins at the top-level Web page for the domain of each facility or system (eg, for the Kaiser Permanente domain, the Web crawler starts at the home page, kaiserpermanente.org), and drills down into successive subpages to build a topographical map of the links within a site. The analytic engine then samples 500 of these subpages and evaluates them based on a battery of assessment items, discussed at length in subsequent sections. A few websites were not assessed by the Web crawler due to technical problems, including timeout due to slow webpages or Web servers, server-side page redirections, or missing or unavailable host names. Only pages residing within the identified facility domain were tested.

To create summarized scores of website performance, the analytic engine scored content along five dimensions: (1) accessibility, (2) content, (3) marketing, (4) technology, and (5) usability. The scores on each dimension are reported on a range from 0 to 10, with a higher score on any given scale representing better comparative performance. The five dimensions detailed in the following sections provide broad assessments of aspects of website quality based on a set of underlying individual metrics. While it is important to note that some specific metrics contribute to more than one of the summarized scales, the scores themselves provide a basis for comparing two or more sites. The definitions of the specific items measured and how they are weighted in the summarized scores are presented in Multimedia Appendix 1.

Accessibility Dimension

Accessibility is a critical factor for reaching as many users as possible, but at-risk groups may not be familiar with access features that require higher levels of computer literacy, such as hovering over highlighted phrases to see additional information. Given the service domain in health care, the issue of accessibility is all the more important, and much discussion has been held on issues of access to services [24,25]. The accessibility score is an assessment of a website’s ease of use for individuals with lower computer literacy levels, including those with physical disabilities that limit their use of a mouse or non-standard browser (such as mobile phones or tablet devices). A number of accessibility evaluation tools are offered by the World Wide Web Consortium (W3C) to explore these issues.

Content Dimension

The content dimension is an assessment of a website’s overall content quality without taking into consideration the technical limitations of the site. Content quality is considered high if the text is grammatically correct, relevant, and updated regularly. The quality of the site’s imagery (ie, photos and graphics) and metadata (ie, information about the data content in specific locations) is also assessed. Elements contributing to the content scale include individual tests of spelling, the degree to which the site adds new material, and the calculated reading age of the text on the pages. In particular, the Flesch-Kincaid readability metrics used in other health-related website studies are included as part of the content analysis [24,25]. The major measures that contribute to the content scale are freshness and the amount of content. The freshness measure is calculated by reading the dates that appear on a website’s pages. Up-to-date content is a positive indicator to consumers that the organization is engaged in state-of-the-art activities. For example, monthly updates to the CEO’s message may be understood to imply that a facility is customer-focused, while out-of-date content may foster a perception that public impressions are less important to the organization. Therefore, routinely adding and changing content to remain current and explicitly documenting the dates that Web pages are updated should be standard practice.

Marketing Dimension

The marketing dimension is an assessment of how readily and reliably information is accessed using search engines, including the appropriateness of content to hyperlinks, the rank and popularity of the website, and other technical aspects related to search engine optimization (SEO). SEO is an important aspect of the marketing scale. As content within a page becomes more accessible to search engines, the organization’s profile in online searches becomes higher. Contributing individual tests include search engine results, search placement, and the use of content keywords that search engines rely on to prioritize websites. Performing these tasks effectively helps health systems maintain a consistent corporate image [26].

Technology Dimension

The technology dimension is an assessment of how well a website is designed, built, and maintained. Technical issues affect the user’s experience and therefore can have a direct impact on the overall utility of the website for making decisions. Elements contributing to the technology index’s scores include website download speed, site structure, code quality, and the use of cascading style sheets to organize content. The technology scale focuses purely on the performance aspects of a website without respect to its content. The major contributor to the score is the speed measure.

Usability Dimension

The usability score is a cross-sectional composite of a number of metrics used in other scales; therefore, it is a composite of metrics, not a composite of the other four scales. This dimension attempts to answer the question of how good a particular website is. Having this at-a-glance metric that rates the overall quality of a website as a single number enables comparisons across a number of critical areas of site presentation. The analytic engine also provides clear information about how each individual organization performs and, by extension, offers clues as to how improvements in these scores might be made.

Average Ranking Across Dimensions

Rank order calculations for each of the five dimensions were averaged to create a single average rank. The average rank score was then calculated across all website domains in order to rank hospitals from 1 to 2407.


This search produced websites associated with 2407 unique domain names. Of those Web domains, 378 were attached to multiple AHA identifiers, indicating that they were part of larger organizations (ie, members in a system). In these cases, the system’s domain was tested once rather than isolating a single facility from its parent organization. In total, 2785 facilities were scored. Organizational characteristics of these facilities are presented in Table 1.

Histograms representing the distribution of observations in each of the summarized scores are presented in Figures 1-5. Across all Figures, we find the scores to have a single mode with skewed distributions. The greatest variance can be seen in Figure 4, Technology. The mean scores for each dimension are presented in Table 2. Looking at the usability scale, the mean score was 5.16 (SD 1.43), with a maximum score of 8 achieved by only five organizations. Mean scores on the other scales ranged between 4.43 (SD 2.19) for technology and 6.49 (SD 0.96) for content.

The top 100 websites on each dimension are presented in Multimedia Appendix 2 with their respective rankings. The last column is an average rank score calculation across the five scales. For instance, jaxhealth.com scored 15th in Accessibility, 61st in Content, 52nd in Marketing, 14st in Technology, and 6th in Usability—resulting in an average score of 29.6, and making it the 1st best site overall. These rank order calculations, along with summary scores across dimensions, are presented for all 2407 websites in Multimedia Appendix 3. Included in these raw data is the AHA ID for each website domain.

Table 1. Hospital characteristics for all US AHAa hospitals by inclusion in the study.b
Hospital characteristicsMatchedNot-matchedTotal US AHAb hospitals
Count of AHA IDs2785.00738.003523.00
Number of births899.731035.24928.12
Adjusted patient days79,030.2682,914.0079,843.82
Transfer-adjusted admissions15,299.7316,595.2915,571.13
Total expenditures152,355,799.64160,273,331.85154,014,368.69
FTE employees1011.651088.711027.80
Number of surgical operations6387.387297.256577.98
Total visits146,552.48149,520.98147,174.32
Number of beds181.04191.52183.24
Average daily census116.40125.38118.28

aAHA: American Hospital Association.

bNot-Matched Hospitals were excluded from the study because they did not have an identifiable Web domain attached to their name.

Table 2. Summary statistics for scales (n=2407).
VariableMean (SD)MinMax
Accessibility5.08 (2.22)0.09.0
Content6.49 (0.96)0.08.6
Marketing5.03 (1.33)0.88.5
Technology4.43 (2.19)0.08.7
Usability5.16 (1.43)0.08.0
Figure 1. Distribution of scores: Accessibility.
View this figure
Figure 2. Distribution of scores: Content.
View this figure
Figure 3. Distribution of scores: Marketing.
View this figure
Figure 4. Distribution of scores: Technology.
View this figure
Figure 5. Distribution of scores: Overall.
View this figure

Principal Results and Implications

Those with an interest in evidence-based management know that one limitation is the presence of data upon which to make management decisions. While a website might have engaging photos and an interesting design, the proper functioning of a website requires far more work “under the hood” than most consider [27]. Layer upon that the importance of a broader social media strategy, and it can be easy to see how a facility might judge their online presence by the attractiveness of the pages. However, health care bears a special burden. With populations that are seeking care when they are least able to deal with externalities, a well-designed site that adheres to national standards and demonstrated best practices is important for issues related to care access [28,29]. Further, the absence of a comprehensive assessment across the entire sector could pose a significant detriment to care, given assumptions that may be made about accessibility and other factors. A thorough assessment requires the auditing of websites beyond the first pages. The present project, systematically assessing the website quality of 2785 hospitals, is a four-fold improvement over prior research and represents what the authors believe to be a census assessment for the online presence of US hospitals and their health systems.

The low mean score we found on the usability scale (5.16 on a 10-point scale, SD 1.43), a measure of general website quality, indicates that organizations’ websites, on average, have significant potential for improvement. In order to make a complete and effective assessment of a health care organization’s website, it is necessary to have the site evaluated using a Web crawling and analytic engine similar to the one employed in the present study. Nevertheless, there are many contributing scale components that organizational leaders can assess by a simple visual inspection of their website (eg, see the 2012 paper by Ford et al [30] for a more comprehensive discussion of this issue). Multimedia Appendix 1 presents a list of scale components with accompanying definitions and the percentage weight of each component within the five scales. These component definitions provide guidance for leaders to identify “low-hanging fruit” to improve their website scores. For example, one could easily search for and repair “Broken Links” (links to Web addresses that do not exist or return an error). Additionally, there is a “Spelling” component that assesses whether the words on a page are spelled correctly, and a “Twitter” component that determines if the website is linked to a Twitter account and how often tweets are posted.

While we did not test for the relationship between organizational type and website performance, a cursory review of the top 100 websites reveals that large facilities performed well. Consumers in urban areas typically go to a local facility for routine care or common procedures, but customers in rural areas may bypass their local facilities. In particular, consumers may travel for specialty care that is inaccessible locally, shopping for a facility and turning to the Internet for information [31,32]. Another explanation for large organizations’ success is that the complex and high-risk nature of cancer and childhood illnesses makes selecting an organization with high-quality ratings and a reputation for innovation particularly important [33,34]. This may contribute to the impetus for specialty facilities to develop high-performing websites for their marketing purposes.

For policymakers, the present analyses of health systems’ websites may provide an indication of whether or not a health system is striving to become an ACO. Given the scope of organizational change required to become an ACO, it stands to reason that a health system’s website would document and reflect such efforts in order to take advantage of that effort in the market. As a result, website quality may be a simple and reliable leading indicator of efforts to make this critical organizational transition. Prior research has found these dimensions of website quality to be linked to important aspects of patient care, including patient safety metrics [35], thus confirming the potential importance of our findings.

Limitations

The authors note several limitations to this study. Facilities associated with an education top domain (.edu) were purposely excluded from the analysis. The decision to exclude the .edu-based hospitals was a difficult one. While these hospitals represent a significant type of player in hospital health care delivery, they also often contain an academic side that would skew assessment. It was therefore impossible to create decision rules to exclude pages that were not directly tied to patient care or the hospital. As a result, inclusion of .edu-based content might result in academic departments unrelated to care influencing measures. It should also be noted that in some instances, hospitals create a facility domain that is separate and apart from the academic center. In these cases, where the domain is .org or .com, we assessed that content, but excluded .edu content if it was linked on any page. Therefore, a facility might have their clinic included, but the information about their doctors residing on the academic side might be excluded. The study authors intend to assess this subgroup in future analyses.

A similar dynamic occurred, at times, with select websites purchased outside of the .com and .org domains. For instance, facilities have also registered on the “.info” and “.us” domains. In these cases, the authors attempted to determine if the identified site was owned by the facility. These were judgment calls on the part of the research team, potentially resulting in counting errors, which would manifest in the descriptive statistics.

Another limitation of the research is related to the emerging nature of health care facilities. With the contraction and centralization of health care, there are fewer independent hospitals. Increasingly hospitals are joining networks and systems of care. These dynamics are becoming more pronounced in the wake of ACO development efforts. As a result, we can increasingly expect that access to regional information will be moved further down into Web pages as these systems centralize their marketing and information dissemination functions. Additionally, it should be noted that many facilities keep their content in a secure environment for users of their health systems. As a result, the assessment might incorrectly assess their Web presence. The assessment only reaches what is not secured based on links accessible from the home pages. This would simulate the information available to either non-members or members using only publically available information.

Finally, the authors recognize a concern around size and scope and the potential to misrepresent a single score as sufficient explanation of information given the diverse nature of facilities. In some cases, a website for a facility can be 6 pages. In another case, the system’s website is over 10,000 pages and centrally managed. To then say that a domain is scored at 5.4 overall for both does not, in fact, mean they are equivalent. There are tradeoffs that any single measure must make. As a result, we have chosen to publish not only the overall score, but also the other tailored scores. The result is a greater nuance, but moves away from that single measure.

Comparison With Prior Work

In 2011, the websites of 636 US hospitals and health systems were tested using a similar methodology, with an overall mean score of 6.37 [30]. While we acknowledge that this prior work displayed higher scores, the addition of so many new facilities makes comparisons to prior evaluations problematic. The present study assessed 2407 domains that covered 2785 facilities. This is a four-fold improvement in prior research and represents what the authors believe to be a census assessment for the online presence of US hospitals and their health systems. We expect smaller facilities to have lower scores, on average. Put another way, we expect a lowering of scores, in comparison to scores calculated from prior research, as a result of greater inclusivity in our current assessment. Future iterations of this study will permit greater comparability across years.

Conclusions

The current analysis presents a significant update to the systematic assessment of hospital social media presence. Given the movement toward having health systems serve as ACOs that can empower consumers [14,30], the number of poorly performing facilities across all the calculated scores is concerning in the near term. The social media and Web presence of many of these organizations represents the first contact health care consumers make with the organization. If such contact fails to make a positive impression on the consumer, alternatives may be explored. In saturated markets where several organizations’ services are interchangeable, a strong and well-designed Web and social media presence can be the difference between patients taking the first step into a facility or doing everything they can to avoid it. Health organizations should strive to standardize the quality of information presented on their websites [36], but they should also take care to deal with issues of accessibility, standards compliance, and search engine optimization.

Conflicts of Interest

The authors are faculty employed by their respective universities, some of which maintain a Web presence that was assessed in this study.

Multimedia Appendix 1

Scale components and weightings.

PDF File (Adobe PDF File), 285KB

Multimedia Appendix 2

Ranking of the Top 100 websites for each dimension and an average ranking across dimensions.

PDF File (Adobe PDF File), 394KB

Multimedia Appendix 3

Complete dataset including domains, associated composite scores, and rankings. (Crosswalk data provided in separate sheet needs to be merged to use the dataset. Not provided merged due to a concern that univariate analysis of the merged data will multiply count single observations, because facilities may share a domain).

XLSX File (Microsoft Excel File), 469KB

  1. Reid P, Borycki EM. Emergence of a new consumer health informatics framework: introducing the healthcare organization. Stud Health Technol Inform 2011;164:353-357. [Medline]
  2. Szokan N. The Washington Post. 2011 Feb 01. Health information remains high on the list of popular uses for the Internet   URL: http://www.washingtonpost.com/wp-dyn/content/article/2011/01/31/AR2011013104786.html [accessed 2014-02-16] [WebCite Cache]
  3. Alpay L, Overberg RI, Schonk BZ. Empowering citizens in assessing health related websites: A driving factor for healthcare governance. International Journal of Healthcare Technology & Management 2007 Jan 01;8(1):141-160 [FREE Full text]
  4. Revere L, Robinson L. How healthcare organizations use the Internet to market quality achievements. J Healthc Manag 2010;55(1):39-49; discussion 49. [Medline]
  5. Coile RC. E-health: reinventing healthcare in the information age. J Healthc Manag 2000;45(3):206-210. [Medline]
  6. Randeree E, Rao HR. E-health and assurance: curing hospital websites. Int J Electron Healthc 2004;1(1):33-46. [Medline]
  7. Wen KY, Kreps G, Zhu F, Miller S. Consumers' perceptions about and use of the Internet for personal health records and health information exchange: analysis of the 2007 Health Information National Trends Survey. J Med Internet Res 2010;12(4):e73 [FREE Full text] [CrossRef] [Medline]
  8. Mittler J, Volmar KM, Shaw BW, Christianson JB, Scanlon DP. Using websites to engage consumers in managing their health and healthcare. Am J Manag Care 2012 Sep;18(6 Suppl):s177-s184 [FREE Full text] [Medline]
  9. Liang C, Chen H. A study of the impacts of website quality on customer relationship performance. Total Quality Management & Business Excellence 2009;20(9):971-988.
  10. Rutitis D, Batraga A, Muizniece L, Ritovs K. Management of corporate identity dimensions in the health care. Procedia - Social and Behavioral Sciences 2012 Oct;58:995-1003. [CrossRef]
  11. Weaver JB, Mays D, Weaver SS, Hopkins GL, Eroglu D, Bernhardt JM. Health information-seeking behaviors, health indicators, and health risks. Am J Public Health 2010 Aug;100(8):1520-1525. [CrossRef] [Medline]
  12. LaPenna AM. New sources of revenue in the age of connectivity. J Healthc Manag 2009;54(1):11-14. [Medline]
  13. O'Donnell J, Hannan EL. An analysis of case-intensity-adjusted nursing staff patterns in New York State skilled nursing facilities. Med Care 1983 Jan;21(1):82-91. [Medline]
  14. Simborg DW. Consumer empowerment versus consumer populism in healthcare IT. J Am Med Inform Assoc 2010;17(4):370-372 [FREE Full text] [CrossRef] [Medline]
  15. Oermann MH, McInerney SM. An evaluation of sepsis Web sites for patient and family education. Plast Surg Nurs 2007;27(4):192-196. [CrossRef] [Medline]
  16. Oermann MH, Lesley ML, VanderWal JS. Using Web sites on quality health care for teaching consumers in public libraries. Qual Manag Health Care 2005;14(3):188-195. [Medline]
  17. Oermann MH, Lowery NF, Thornley J. Evaluation of Web sites on management of pain in children. Pain Manag Nurs 2003 Sep;4(3):99-105. [Medline]
  18. The Agency for Healthcare Research and Quality. Assessing the Quality of Internet Health Information: Summary. 1999.   URL: http://www.ahrq.gov/research/data/infoqual.html [accessed 2014-02-16] [WebCite Cache]
  19. W3C Standards. 2013.   URL: http://www.w3.org/standards/ [accessed 2014-02-16] [WebCite Cache]
  20. Williams P, Nicholas D, Huntington P, McLean F. Surfing for health: user evaluation of a health information website. Part one: Background and literature review. Health Info Libr J 2002 Jun;19(2):98-108. [Medline]
  21. Criteria for Assessing the Quality of Health Information on the Internet. 2002.   URL: http://www.apha.org/NR/exeres/412DDD11-42A0-46CB-8705-F9D47A073AF1.htm [accessed 2014-02-16] [WebCite Cache]
  22. Ivory MY. Automated Web site evaluation: researchers' and practitioners' perspectives. In: Human-Computer Interaction Series. The Netherlands: Kluwer Academic Publishers; 2003.
  23. Chiang MF, Starren J. Evaluation of consumer health website accessibility by users with sensory and physical disabilities. Stud Health Technol Inform 2004;107(Pt 2):1128-1132. [Medline]
  24. Stewart MJ, Georgiou A, Westbrook JI. Successfully integrating aged care services: a review of the evidence and tools emerging from a long-term care program. Int J Integr Care 2013;13:e003 [FREE Full text] [Medline]
  25. Clarke S, Calam R, Morawska A, Sanders M. Developing web-based Triple P 'Positive Parenting Programme' for families of children with asthma. Child Care Health Dev 2013 May 12:1. [CrossRef] [Medline]
  26. Batraga A, Rutitis D. Corporate identity with the health care industry. Economics and Management 2012 Dec 07;17(4):1545-1551. [CrossRef]
  27. Griffiths KM, Christensen H. Website quality indicators for consumers. J Med Internet Res 2005;7(5):e55 [FREE Full text] [CrossRef] [Medline]
  28. Beckjord EB, Finney Rutten LJ, Squiers L, Arora NK, Volckmann L, Moser RP, et al. Use of the internet to communicate with health care providers in the United States: estimates from the 2003 and 2005 Health Information National Trends Surveys (HINTS). J Med Internet Res 2007;9(3):e20 [FREE Full text] [CrossRef] [Medline]
  29. Zeng X, Parmanto B. Web content accessibility of consumer health information Web sites for people with disabilities: a cross sectional evaluation. J Med Internet Res 2004 Jun 21;6(2):e19 [FREE Full text] [CrossRef] [Medline]
  30. Ford EW, Huerta TR, Schilhavy RA, Menachemi N. Effective US health system websites: establishing benchmarks and standards for effective consumer engagement. J Healthc Manag 2012;57(1):47-64; discussion 64. [Medline]
  31. Saunders C, Bellamy GR, Menachemi N, Chukmaitov AS, Brooks RG. Bypassing the local rural hospital for outpatient procedures. J Rural Health 2009;25(2):174-181. [CrossRef] [Medline]
  32. Damman OC, van den Hengel YK, van Loon AJ, Rademakers J. An international comparison of web-based reporting about health care quality: content analysis. J Med Internet Res 2010;12(2):e8 [FREE Full text] [CrossRef] [Medline]
  33. Kim K, Kwon N. Profile of e-patients: analysis of their cancer information-seeking from a national survey. J Health Commun 2010 Oct;15(7):712-733. [CrossRef] [Medline]
  34. Kinnane N. Evaluation of a hospital-based cancer information and support centre. Support Care Cancer 2012 Feb;20(2):287-300. [CrossRef] [Medline]
  35. Ford E, Huerta TR, Schilhavy RA, Menachemi N. Effective US health system websites: establishing benchmarks and standards for effective consumer engagement. J Healthc Manag 2012;57(1):47-64; discussion 64. [Medline]
  36. Suchy K. A lack of standardization: the basis for the ethical issues surrounding quality and performance reports. J Healthc Manag 2010;55(4):241-251. [Medline]


ACO: accountable care organization
AHA: American Hospital Association
SEO: search engine optimization


Edited by G Eysenbach; submitted 28.10.13; peer-reviewed by A Kazley, A Vagelatos; comments to author 23.12.13; revised version received 26.12.13; accepted 09.01.14; published 25.02.14

Copyright

©Timothy R Huerta, Jennifer L Hefner, Eric W Ford, Ann Scheck McAlearney, Nir Menachemi. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 25.02.2014.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.