Published on 16.11.09 in Vol 11, No 4 (2009): Oct-Dec
Age-Sensitive Design of Online Health Information: Comparative Usability Study
Background: Older adults’ health maintenance may be enhanced by having access to online health information. However, usability issues may prevent older adults from easily accessing such information. Prior research has shown that aging is associated with a unique pattern of cognitive changes, and knowledge of these changes may be used in the design of health websites for older adults.
Objective: The goal of the current study was to examine whether older adults use of a health information website was affected by an alternative information architecture and access interface (hierarchical versus tag-based).
Methods: Fifty younger adults (aged 18-23) and 50 older adults (aged 60-80) navigated a health information website, which was organized hierarchically or used tags/keywords, to find answers to health-related questions while their performance was tracked. We hypothesized that older adults would perform better in the tag-based health information website because it placed greater demands on abilities that remain intact with aging (verbal ability and vocabulary).
Results: The pattern of age-related differences in computer use was consistent with prior research with older adults. We found that older adults had been using computers for less time (F1,98= 10.6, P= .002) and used them less often (F1,98= 11.3, P= .001) than younger adults. Also consistent with the cognitive aging literature, younger adults had greater spatial visualization and orientation abilities (F1,98= 34.6, P< .001 and F1,98= 6.8, P= .01) and a larger memory span (F1,98= 5.7, P= .02) than older adults, but older adults had greater vocabulary (F1,98= 11.4, P= .001). Older adults also took significantly more medications than younger adults (F1,98= 57.7, P< .001). In the information search task, older adults performed worse than younger adults (F1,96= 18.0, P< .001). However, there was a significant age × condition interaction indicating that while younger adults outperformed older adults in the hierarchical condition (F1,96= 25.2, P< .001), there were no significant age-related differences in the tag-based condition, indicating that older adults performed as well as younger adults in this condition.
Conclusions: Access to online health information is increasing in popularity and can lead to a more informed health consumer. However, usability barriers may differentially affect older adults. The results of the current study suggest that the design of health information websites that take into account age-related changes in cognition can enhance older adults’ access to such information.
J Med Internet Res 2009;11(4):e45
According to a recent Pew report on Internet usage, up to 80% of American Internet users have accessed health information on the Internet, with 64% of Americans searching for information about a specific disease and 51% searching for treatments . The ease with which health consumers can access high-quality, doctor-reviewed medical information has the potential to allow patients to take more control of their health outcomes. The same Pew report shows mostly positive perceived outcomes with this ease of information access, such as feeling relieved or comforted by the information found (56%) and, more importantly, feeling confident about raising questions to the doctor after searching the Internet (56%).
However, the wealth of available information may be a curse to some health consumers: 25% of consumers who have searched for health information on the Internet felt overwhelmed by the amount of information available . In addition, some sources of online health information may be of higher quality than others [ ]. The issue of quality and reliability of information is problematic for interpretation when combined with health consumers’ potential lack of knowledge on health topics. Despite these downsides, access to information does seem to lead to better-perceived outcomes, such as increased confidence in patients’ perception of their health care decision-making ability [ ].
Abilities, Aging, and Internet Use
The current study examined older adults’ ability to easily access online health information. Specifically, we were interested in understanding whether interface design influenced successful access and usage of online health information sources by older adults (those age 60 and over). As a group, older adults are less active users of the Internet and Web services when compared to other age groups . In addition, older adults are also more likely to suffer from various health conditions [ ] and take more medications [ ]. Access to quality health information might be especially beneficial to older adults’ maintenance of their more complicated health situations [ ]. Being better informed may allow them to ask more questions of their health care providers or to alleviate their concerns due to lack of information.
There are a myriad of reasons why older adults are less active users of the Web. One reason is that getting older is associated with cognitive changes that make using the Web and computers in general more challenging [- ]. Using the Internet, and more specifically navigating online information sources, places particularly heavy demands on “fluid abilities” [ ]. Fluid cognitive abilities, for example working memory (the ability to hold contents in memory while attending to other things) and spatial abilities (creating and manipulating mental representations such as maps), allow us to think and act in situations that are novel[ ]. Fluid abilities can be described as the means or process that allows us to learn and adapt in novel situations. When we browse or navigate a website, our performance depends on our ability to keep track of where we were in the system (working memory) and our ability to create abstract maps or models of the system (spatial abilities) [ ]. As these abilities decline with age, performance on tasks that depend heavily on these abilities suffers.
However, increasing age is also associated with an increase in “crystallized intelligence” . Crystallized intelligence is the nonspecific, accumulated knowledge that one gains from a lifetime of education and experience. It is the product of formal education and life experience. It is commonly measured in the laboratory with tests of vocabulary or general knowledge. In a prior study, Pak and Price [ ] examined age-related changes in cognition and designed a Web interface that was adjusted for older users’ cognitive abilities. When websites were designed around keywords, or tags, instead of in a hierarchy (or folders), older users were more efficient at finding information online. In that study, younger and older users browsed a fictional travel information website to answer a series of specific questions (eg, “Where do you mail your passport application?”). Pak and Price theorized that the older adults’ advantage was due in part to their greater facility with general vocabulary and verbal ability and the interface’s reduced demands on age-sensitive spatial abilities. In their analysis, the tag-based interface placed greater demands on knowledge of vocabulary, which is an ability that grows with age; that is, the cognitive requirements of navigating in a tag-based system (compared to navigating a hierarchy) seemed to have been especially dependent on good verbal and vocabulary abilities. In many studies, including our prior study, older adults routinely outperformed younger adults on tests of vocabulary and verbal knowledge [ ].
Overview of the Study
In Pak and Price’s study , the travel domain was chosen to equate the amount of knowledge between younger and older adults. Prior research had shown that younger and older users did not differ in their knowledge and experience with such information [ ]. However, the findings from our prior study needed replication, especially in a domain that may be more applicable to older adults: online health information. In the current study, we examined whether older adults’ navigation of a health information website would improve in a tag-based interface. There currently exists a large body of literature related to aging and usability issues on the Web (see [ ] for an extensive review). However, there is far too little work on translation of this basic research into design and testing of design recommendations to improve Web usability for older adults.
The younger adults recruited for the study were college students, while the older adults were community-dwelling, independent-living adults recruited through newspaper advertisements. Younger adults participated for course credit or US$7/hour, whereas the older adults participated in exchange for US$7/hour.
Participants were asked about their computer experience, including length of computer experience and frequency of use. To indirectly measure experience or exposure to health-related information, participants also reported the amount of prescription medications they were taking at the time of the study. We also had participants fill out a more direct measure of health literacy, the Short Test of Functional Health Literacy in Adults (STOFHLA ). Finally, several ability measures were included to compare our sample of participants to typical samples used in age studies. They were a measure of general vocabulary knowledge (Shipley vocabulary test [ ]), a measure of working memory (reverse digit span [ ]), two measures of spatial ability (paper folding and cube comparison [ ]), and finally a measure a perceptual speed (digit symbol substitution [ ]).
To create the interfaces, we first copied content from various health-related websites. The majority of information was taken from the National Institutes of Health website NIHSeniorHealth . A total of 122 Web pages were captured, and the pages’ appearance was standardized (eg, same font, size, colors). These Web pages were organized using one of two information architectural schemes: tag/keyword-based or hierarchical (folders). To create these information architectures, we first grouped the 122 pages into a hierarchical system. This was done by having undergraduate students carry out a card-sorting procedure. In the procedure, students placed each page into an organizational hierarchy that made sense to them. Afterwards, they named these groupings. The results of several sessions of card sorting were merged to create the hierarchical condition that consisted of 10 top-level categories (bone and joint, cardiovascular diseases, depression, diabetes, dry mouth, hearing and vision, lung diseases, medications, skin cancer, and talking to your doctor). This same methodology was used by Pak and Price [ ]. The tag-based system was created directly from the hierarchy. For example, if a page on gout treatment was organized in the hierarchical condition as Bone & Joint > Arthritis > Gout > Treatment, it was assigned the keywords “bone & joint,” “arthritis,” “gout,” and “treatment.” Our rationale for such a label assignment system was our overriding concern to keep the label names as constant as possible across both conditions. We did not want to inadvertently present more or better information in one condition over another.
The main difference between the two conditions (aside from the visual difference; the hierarchical condition was visually longer) was that Web pages were only accessible in the hierarchical condition if the participant reached the single category in which it resided (ie, they had to reach the exact “folder” or “subfolder” that held the desired page). However, in the tag-based condition, Web pages could be accessed by selecting any label that was associated with the page.and show the access interfaces for the hierarchical and tag-based conditions, respectively. They illustrate the task flow for a user answering a question about gout. In both conditions, users were presented with the question at the top of the screen, the navigation interface (hierarchical or tag-based) on the left side, and the information pane on the right side.
Hierarchical organization is typically how information is organized on the Web (eg, NIHSeniorHealth). A sample hierarchical organization in our example would be the pages related to “gout” organized within the “arthritis” folder, which is within the “bones & joint” folder. The hierarchical organization is identical to how one might organize files on a computer using nested folders.
In the tag-based organizational scheme, Web pages were each labeled with keywords and the interface presented these keywords to the user. For example, if the user clicked on the gout keyword, all pages that were pre-assigned that keyword appeared (eg, a page on gout prevention, definition of gout). This is similar to how photographs are organized on the photo-sharing website Flickr.com. However, the important difference was that in our website the tags were pre-assigned by the experimenters, while on other websites tags are user generated.
Participants were asked to answer health-related questions by searching the health information site that we provided. An example question was, “What is considered normal blood pressure?” Half of the participants searched the hierarchically organized site, while the other half searched the tag-based site. When they found the answer on a specific Web page, they clicked the answer text on the page and the application presented feedback as to the correctness of the answer. A total of 25 questions were presented. The Web application was programmed in the PHP scripting language and ran on a local Web server. The application recorded the name of each visited Web page, how much time was spent on each page, and the number of times participants clicked the back button.
Study Design and Procedure
The study was a 2 (age group: young, old) × 2 (organizational scheme: hierarchical or tag-based) factorial with age group as a grouping variable and organizational scheme as a between-group variable. Sessions were assigned to each condition and tested in a computer lab in groups of three to four people so that all participants in a session were assigned to either a hierarchical condition or a tag-based condition. The dependent variables were task completion time, errors, and mouse clicks.
Participants first filled out paperwork (eg, consent form, demographics, abilities tests) and then moved to the computer to start the information search task. Participants as a group were first guided by the experimenter through two example questions. During the example questions, participants got acquainted with the interface, and any questions were answered. After this practice session, participants were instructed to complete the search tasks as quickly but as accurately as possible and were left to complete the task on their own. These instructions were reiterated by the computer after every trial.
The three dependent measures of performance (task time, clicks, and errors) were subjected to a multivariate analysis of variance (MANOVA) with condition and age group as between-subject factors. In addition to an analysis of the individual dependent measures, a composite variable with the three dependent measures was created. The benefits of creating a composite are increased stability of measurement. A composite performance variable incorporating task completion time, error rate, and number of steps was created for analysis. Each dependent variable was normalized (zscore transformed), and these individual zscores were averaged to create a unit-less composite performance variable that ranged from 1 (worst performance) to −1 (best performance). The analysis of composite variables in usability evaluation has been suggested by researchers as a way to increase stability of measurement  and ease interpretation [ , ]. This composite performance measure was subjected to a 2 × 2 analysis of variance (ANOVA).
Fifty younger adults (29 female) ranging in age from 18 to 23 (mean = 19.5, SD = 1.7) and 50 older adults (27 female) ranging in age from 60 to 80 (mean = 70.6, SD = 5.6) participated in the study.
There were significant age group differences in total length of computer experience and frequency of use, with younger adults using computers for a longer period of time than older adults (F1,98= 10.6, P = .002, ηp2= .10) and more frequently (F1,96= 11.3, P = .001, ηp2= .1). Older adults reported taking significantly more prescriptions than younger adults (F1,98= 57.7, P < .001, ηp2= .37). According to the STOFHLA, there were no age group differences in health literacy. Older adults typically outperform younger adults on tests of vocabulary, and this was the case in our sample, with the older adults outperforming the younger adults (F1, 98= 11.4, P= .001, ηp2= .11). Younger adults, however, outperformed the older adults in the fluid ability measures (memory span F1,98= 5.7, P= .02, ηp2= .06; spatial visualization F1,98= 34.6, P< .001, ηp2= .27; spatial orientation F1,98= 6.8, P= .01, ηp2= .07; and perceptual speed F1,98= 70.2, P< .001, ηp2= .42), which is consistent with the general literature on aging and cognition .
General participant characteristics are presented in.
Performance (mean task completion times, mean number of mouse clicks to completion, and mean errors per task) is shown inand illustrated in .
The results of the analysis of dependent measures showed significant overall main effects of condition and age group (F1,94= 23.8,P< .001, ηp2= .43, F1,94= 29.5, P< .001, ηp2= .48, respectively), and the interaction was significant (F1,94= 4.7, P< .001, ηp2= .13), so the main effects were not followed up. Follow-up analysis revealed that the significant interaction was due to a significant interaction in mean clicks (F1,96= 8.4, P< .001, ηp2= .04) and mean errors (F1,96= 12.29, P< .001, ηp2= .11). The interaction in mean task time was not significant (P= .06). The source of the interaction in mean clicks and errors was not significant age difference in clicks or errors in the tag condition but in the taxonomy (hierarchical) condition: older adults made more clicks than younger adults. These interactions are illustrated in. These results conceptually replicate the earlier results [ ] that showed that navigation through a website can be improved if the organization is organized around keywords, not a hierarchy.
The analysis of the composite measure of performance showed that the main effect of age group was significant (F1,96= 18.0, P <.001, ηp2= .16), indicating that older adults in general performed worse than younger adults. There was no significant main effect of condition; however, the age group × condition interaction was significant (F1,96= 8.1, P = .005, ηp2= .08), indicating that condition (hierarchical vs tag-based) differentially affected each age group. Post hoc analyses (Bonferonni method) revealed that the source of this interaction was the hierarchical condition. Younger adults significantly outperformed older adults in the information search task when using the hierarchical condition (F1,96= 25.2, P <.001, ηp2= .21). However, in the tag-based condition, there were no significant performance differences between the younger and older adults in the information search task. This replicates our earlier findings that a tag-based system that relies on the generation and recognition of keywords may benefit older adults’ information-finding performance. The interaction is represented in.
The current finding that older users’ performance searching for medical information through a tag-based system was better than through a hierarchically organized system replicates our earlier findings examining travel information. However, we also wanted to examine how pre-existing domain knowledge or experience with health and medical information might be related to performance in each of the two organizational schemes.shows the correlations between performance, number of medications, and experience measures (computer experience and health/medical experience).
In the hierarchically organized condition, performance was significantly correlated with age (older adults performed worse than younger adults), the number of medications currently taken (increased number of medications was associated with worse performance), and health literacy (those with greater health literacy performed better). It is somewhat intuitive that health literacy (STOFHLA) was associated with better performance in the information search task, but it is puzzling that performance was not correlated with number of medications taken. We initially assumed that number of medications could be used as an indirect indicator of health knowledge, with the assumption that people who took more medications would have a greater amount of health knowledge because of their need to manage and understand their medication regimen. However, the correlation between STOFHLA and number of medications was not significant. One possibility could be that number of medications taken is a better indicator of overall cognitive and physical health status (with those taking more medications having worse health and thus cognitive status), and not knowledge.
In contrast, the pattern of correlations in the tag-based system show that performance was not significantly associated with age, number of medications, or health knowledge. These results are consistent with those of Pak and Price , which showed that age was not a significant predictor of performance in the tag-based condition but did predict significant variance in the hierarchical condition. The implication is that hierarchical systems are sensitive to age (and thus age-related differences in abilities and knowledge), while tag-based systems are not.
The online health information consumer ranges widely in age, experience with computers, and health status [, ]. It is thus critical to examine usability issues that assure people with varying backgrounds can successfully access online health information. The goal of the current study was to determine whether older adults’ health information search and retrieval performance could be improved with relatively modest interface usability changes based on the cognitive aging literature. Earlier work showed that when older adults searched through a tag-based website, their performance was relatively better than when they searched through a hierarchically organized website. The basis for improvement was theorized to be that tag-based interfaces, compared to hierarchical organizations, shift cognitive demands from spatial abilities (ie, knowing where you are) to verbal/vocabulary abilities (knowing keywords).
The results from the current study show that with a restructuring of the information architecture and the access interface (from a hierarchical organization to one based on keywords or tags), older adults are able to improve their information search and retrieval performance. This was presumably the result of a change in the ability demands of each system, with the tag-based system placing less stringent demands on older adults’ cognition than the hierarchical system. This study represents the first replication of our earlier results, but with a topical domain that may be especially relevant to older adults: online health information.
While older adults’ performance was improved in a tag-based interface, younger adults’ performance was worse in the tag-based system. This may be due to the simultaneous reduced spatial ability demands and increased verbal/knowledge demands from the tag-based interface coupled with younger adults’ relative lack of verbal abilities (compared to older adults). This suggests the possibility that, for optimal performance, online health information providers may need to provide age-specific interfaces for their users. However, it also stresses the need for further research into interfaces that can combine the beneficial aspects of tag-based and hierarchical interfaces useful for people of all ages. One such novel interface, faceted navigation, is being used on some websites and may be the bridge between hierarchical interfaces and pure tag-based interfaces . In facetted navigation systems, users progressively narrow search results by selecting facets, or dimensions, of information. For example, a free-text search of diabetes from a health information site might return several thousand results. In a facetted navigation system, the user could then focus on facets of those results (eg, only news articles, or peer-reviewed research) by selecting keywords. The user experience is similar to tag-based interfaces, but the difference is that the presentation of tags can be hierarchical.
One limitation of this research is that online medical information websites or portals may only be one of several possible destinations for health consumers [, ]. For example, information searchers may turn to generic search engines (eg, Google, Yahoo), which brings unique usability problems [ , ]. In a usability study of Web-based health information, Eysenbach and Köhler found that older users exhibited suboptimal search engine use such as viewing only the initial search results page (when more were available) [ ]. Another limitation is related to the design of the materials. Because of our desire to keep the hierarchy and tag-based conditions as conceptually consistent as possible, the hierarchy (presented on the left-hand side) was slightly visually taller than the tag-based condition. However, we believe that this difference was minor and is unlikely to, by itself, explain the observed effects. For example, it could be the case that the longer hierarchy (and concomitant increase in visual clutter) may explain older adults performing more poorly in the hierarchical condition compared to the tag-based condition. However, the clutter explanation does not explain why younger adults performed relatively better in the presence of the hierarchical condition compared to tag-based—they would also be subject to clutter effects.
The current research was an attempt to add evidence-based knowledge to the problem of older adult information searching on the Web. There is now nearly a critical mass of literature on age-related changes as they may relate to the use of Web resources , but much work is necessary to translate this basic knowledge of age differences into specific design recommendations and recommendations for older adults.
This research was partially supported by a Google Research Award to Richard Pak.
Conflicts of Interest
- Fox S, Fallows D. Internet health resources: health searches and email have become more commonplace, but there is room for improvement in searches and overall Internet access. Washington, DC: Pew Internet & American Life Project; 2003. URL: http://www.pewinternet.org/pdfs/PIP_Health_Report_July_2003.pdf%20 [WebCite Cache]
- Greenberg L, D'Andrea G, Lorence D. Setting the public agenda for online health search: a white paper and action agenda. J Med Internet Res 2004 Jun 8;6(2):e18 [FREE Full text] [Medline] [CrossRef]
- Fox S. Digital divide: There are clear differences among those with broadband connections, dial-up connections, and no connections at all to the Internet. Washington, DC: Pew Internet & American Life Project; 2005. URL: http://www.pewinternet.org/PPF/r/165/report_display.asp [WebCite Cache]
- Meadows M. FDA Consumer Magazine. 2006 . Medication use and older adults URL: http://www.fda.gov/fdac/features/2006/406_olderadults.html [WebCite Cache]
- ; National Center for Health Statistics. Health, United States, 2007 with Chartbook on Trends in the Health of Americans. Hyattsville, MD: National Center for Health Statistics; 2007. URL: http://www.cdc.gov/nchs/data/hus/hus07.pdf#063 [WebCite Cache]
- Sinden DS, Wister AV. E-health promotion for aging baby boomers in North America. Gerontechnology 2008;7(3):271-278. [CrossRef]
- Czaja SJ, Charness N, Fisk AD, Hertzog C, Nair SN, Rogers WA, et al. Factors predicting the use of technology: findings from the Center for Research and Education on Aging and Technology Enhancement (CREATE). Psychol Aging 2006 Jun;21(2):333-352 [FREE Full text] [Medline] [CrossRef]
- Hawthorn D. Possible implications of aging for interface designers. Interact Comput 2000;12(5):507-528. [CrossRef]
- Garfein AJ, Schaie KW, Willis SL. Microcomputer proficiency in later-middle-aged and older adults: Teaching old dogs new tricks. Soc Behav 1988;3(2):131-148.
- Pak R, Czaja SJ, Sharit J, Rogers WA, Fisk AD. The role of spatial abilities and age in performance in an auditory computer navigation task. Comput Human Behav 2006;24(6):3045-3051 [FREE Full text] [Medline] [CrossRef]
- Horn JL, Cattell RB. Age differences in fluid and crystallized intelligence. Acta Psychol (Amst) 1967;26(2):107-129. [Medline] [CrossRef]
- Pak R, Price MM. Designing an information search interface for younger and older adults. Hum Factors 2008 Aug;50(4):614-628. [Medline] [CrossRef]
- Pak R, Stronge AJ, Padgett L. Reviewing website usability: Information organization. 2002 Presented at: The 13th Annual Southeastern Regional Student Convention in Gerontology and Geriatrics; April 2002; Athens, GA, USA.
- Arch A. Web Accessibility for Older Users: A Literature Review. W3C Working Draft. 2008 [FREE Full text] [WebCite Cache]
- Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test to measure functional health literacy. Patient Educ Couns 1999 Sep;38(1):33-42. [Medline] [CrossRef]
- Shipley W. Shipley Institute of Living Scale. Los Angeles, CA: Western Psychological Press; 1986.
- Wechsler D. Wechsler Adult Intelligence Scale. 3rd edition. San Antonio, TX: The Psychological Corporation; 1997.
- Ekstrom RB, French JW, Harman HH, Dermen D. Manual for Kit of Factor-Referenced Cognitive Tests. Princeton, NJ: Educational Testing Service; 1976.
- Dillon WR, McDonald R. Methodological and statistics concerns of the experimental behavioral researcher. J Consum Psychol 2001;10:62-64.
- National Institutes of Health. NIHSeniorHealth URL: http://nihseniorhealth.gov/ [accessed 2009-04-24] [WebCite Cache]
- Sauro J, Kindlund E. A method to standardized usability metrics into a single score. New York: Association for Computing Machinery; 2005 Presented at: CHI 2005; April 2-7, 2005; Portland, OR, USA URL: http://www.webcitation.org/5gNJGsEwF
- Houston TK, Allison JJ. Users of Internet health information: differences by health status. J Med Internet Res 2002 Nov 22;4(2):E7 [FREE Full text] [Medline] [CrossRef]
- Hearst M. Clustering versus faceted categories for information exploration. Commun ACM 2006;49(4):59-61. [CrossRef]
- Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ 2002 Mar 9;324(7337):573-577 [FREE Full text] [Medline] [CrossRef]
- Stronge AJ, Rogers WA, Fisk AD. Web-based information search and retrieval: effects of strategy use and age on search success. Hum Factors 2006;48(3):434-446. [Medline] [CrossRef]
|STOFHLA: Short Test of Functional Health Literacy in Adults|
Edited by G Eysenbach; submitted 18.02.09; peer-reviewed by A McLaughlin, D Billman, R Appleyard PhD BA (Oxon), M Ziefle; comments to author 19.03.09; revised version received 12.08.09; accepted 03.09.09; published 16.11.09
© Richard Pak, Margaux M Price, Jason Thatcher. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 16.11.2009.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.