Advertisement: Preregister now for the Medicine 2.0 Congress
The Digital Divide Among Low-Income Homebound Older Adults: Internet Use Patterns, eHealth Literacy, and Attitudes Toward Computer/Internet Use
Namkee G Choi, PhD; Diana M DiNitto, PhD
The University of Texas at Austin, Austin, TX, United States
The University of Texas at Austin
1925 San Jacinto Blvd
Austin, TX, 78712-0358
Phone: 1 512 232 9590
Fax: 1 512 471 9600
Background: Internet technology can provide a diverse array of online resources for low-income disabled and homebound older adults to manage their health and mental health problems and maintain social connections. Despite many previous studies of older adults’ Internet use, none focused on these most vulnerable older adults.
Objective: This study examined Internet use patterns, reasons for discontinued use, eHealth literacy, and attitudes toward computer/Internet use among low-income homebound individuals aged 60 and older in comparison to their younger counterparts—homebound adults under age 60.
Methods: Face-to-face or telephone surveys were conducted with 980 recipients of home-delivered meals in central Texas (78% were age 60 years and older and 22% under age 60). The eHealth Literacy Scale (eHEALS) and the efficacy and interest subscales of the Attitudes Toward Computer/Internet Questionnaire (ATC/IQ) were used to measure the respective constructs. Age groups were compared with chi-square tests and t tests. Correlates of Internet use were analyzed with multinomial logistic regression, and correlates of eHEALS and ATC/IQ scores were analyzed with OLS regression models.
Results: Only 34% of the under-60 group and 17% of the 60 years and older group currently used the Internet, and 35% and 16% of the respective group members reported discontinuing Internet use due to cost and disability. In addition to being older, never users were more likely to be black (OR 4.41; 95% CI 2.82-6.91, P<.001) or Hispanic (OR 4.69; 95% CI 2.61-8.44, P<.001), and to have lower incomes (OR 0.36; 95% CI 0.27-0.49, P<.001). Discontinued users were also more likely to be black or Hispanic and to have lower incomes. Among both age groups, approximately three-fourths of the current users used the Internet every day or every few days, and their eHEALS scores were negatively associated with age and positively associated with frequency of use. Among the 60 and older group, a depression diagnosis was also negatively associated with eHEALS scores. ATC/IQ efficacy among never users of all ages and among older adults was positively associated with living alone, income, and the number of medical conditions and inversely associated with age, Hispanic ethnicity, and Spanish as the primary language. Although ATC/IQ interest among older adults was also inversely associated with age, it was not associated with Hispanic ethnicity and Spanish as the primary language.
Conclusions: This study is the first to describe in detail low-income disabled and homebound adults’ and older adults’ Internet use. It shows very low rates of Internet use compared to the US population, either due to lack of exposure to computer/Internet technology; lack of financial resources to obtain computers and technology; or medical conditions, disabilities, and associated pain that restrict use. Recommendations to reduce the digital divide among these individuals are provided.
(J Med Internet Res 2013;15(5):e93)
homebound older adults; housebound older adults; Internet; eHealth literacy; attitudes toward Internet
With the unprecedented growth of the aging population, the number of disabled and homebound older adults continues to increase. Census data for 2009 show that 23.5% of the 40 million noninstitutionalized adults aged 65 years and older in the United States had an ambulatory disability and 15.8% had an independent living disability . The Centers for Medicare and Medicaid Services also report that in 2011, 3.3 million Medicare or Medicaid recipients aged 65 and older received home health care services . Of disabled and homebound older adults, the oldest, poorest, and physically and mentally most vulnerable tend to receive home-delivered meals (HDM, commonly known as “Meals on Wheels”) funded under Title III of the Older Americans Act . In 2009, a little over 880,000 older adults (9%, 60-64 years old; 22%, 65-70 years old; 40%, 75-84 years old; and 30%, 85 years old and up) received HDM [4,5]. These low-income homebound older adults, who often have multiple chronic medical conditions and disabilities, face daunting tasks maintaining their independent living status. Their mobility impairments and lack of financial resources are significant barriers to conducting basic activities of daily living and instrumental activities of daily living (ADLs/IADLs) and remaining socially engaged, and as a result, they also become highly vulnerable to depression [6-8].
Internet technology can provide a diverse array of online resources for homebound older adults to manage their health and mental health problems and ADLs/IADLs. For example, they can use the Internet to search for health information; participate in freely available online chronic disease self-management programs, health and mental health support groups, and exercise programs; and order medications, make appointments, and communicate with their health care providers. Homebound older adults may also be able to stay connected to their close social support networks and the larger community via emails and video calls and by visiting religious sites, social networking sites, chat/discussion groups, interest/hobby groups, and news and blog sites. They may also take advantage of online shopping, banking, and bill paying to take more control of their daily lives. Furthermore, Internet access can allow them to benefit from the growing number of telehealth and telemental health interventions [9-11].
Older-adult Internet users indeed report multiple benefits of using this technology including convenience of accessing health-and nonhealth-related information; increased communication and social connections with family, friends, and others regardless of geographical distance; keeping abreast of news and other happenings in their immediate and global communities; participating in a variety of online educational, social, and recreational activities; and convenience of online shopping, banking, and travel arrangements [12-18]. Internet technology and web-based resources may likewise promote homebound older adults’ physical and mental health and reduce their social isolation and dependence on informal and formal support systems.
Although older adults are the fastest growing group of Internet users, their use still lags behind all other age groups. In April 2012, the Pew Internet and American Life Project found that among a nationally representative sample of Americans, 53% of those aged 65 and older, but only 34% of those aged 76 and older, used the Internet or email, in contrast to 97% of the 18-29 age group, 91% of the 30-49 age group, and 77% of the 50-64 age group . Given their advanced age, high degree of functional impairment, limited financial resources, and social isolation, Internet use among low-income homebound older adults is likely to be even lower than that among general older adults. In fact, previous studies found that older adults who did not use the Internet or email tended to be older and of racial/ethnic minority status and had less education, worse physical and functional health, fewer social and financial resources, and greater loneliness/perceived social isolation [20-25]. Other studies also report that the most powerful predictors of not using information technology among older adults are cognitive decline associated with aging processes and attitudes such as anxiety about computer use and the perception that the technology was not useful for them [14,25,26]. Previous studies have not found consistent evidence about depressive symptoms as a correlate of older adults’ computer/Internet use. One study found depressive symptoms to be negatively associated with Internet use ; another found no relationship .
Older adults themselves report the following reasons for not using computer/Internet technology: the cost of the computer/other equipment and Internet access, functional impairments such as arthritis and joint pain that interfere with typing, visual deficits, ergonomic barriers (eg, small font sizes), lack of computer knowledge, lack of computer-efficacy (beliefs about their ability to use computers/Internet technology) and general self-efficacy (eg, “too old to learn new things”), and mistrust of Internet systems and privacy-related concerns [15,21,28]. However, previous studies also found that experience with computers/Internet reduced anxiety and increased self-confidence and positive attitudes about computers/Internet use in older adults regardless of income and educational levels [18,29-32].
For older adults in general and for low-income, disabled, homebound older adults in particular, eHealth literacy, along with Internet access, is an important dimension of the digital divide, ie, those who use versus those who do not use this technology. eHealth literacy is “the ability to seek, find, understand and appraise health information from electronic sources and apply knowledge gained to addressing or solving a health problem” and is composed of basic literacy, health literacy, scientific literacy, media literacy, and computer literacy . Previous studies show that (1) eHealth literacy is lower among older adults, those with lower socioeconomic status, and those with less computer experience, and (2) higher eHealth literacy is associated with more positive outcomes from Internet searches in three domains: cognitive (eg, health knowledge/information gathering), instrumental (eg, self-management of health needs and health behaviors), and interpersonal (eg, interactions with physicians) [34-36]. Studies also find that health information technology training among older adults results in significant gains in eHealth literacy and ability to navigate complex health websites [18,34]. Government agencies and many private nonprofit and for-profit sector organizations now make prodigious amounts of health and mental health information and resources available online. Given vulnerable low-income homebound older adults’ substantial health and mental needs, examining their ability to search for high-quality health information/resources and make informed decisions about applying the information to improve their quality of life may be particularly useful. This examination could identify their training needs to help close the digital divide and allow them to reap the multiple benefits of Internet and computer technology use.
Given that no study has examined Internet use patterns, eHealth literacy, and attitudes toward computer/Internet among low-income homebound older adults (aged 60 and older), we did so by comparing them with their younger counterparts—low-income homebound adults under age 60. The research questions in the present study were: Compared to their younger counterparts, (1) What are the rates of current Internet use, previous use, and never use among older adults?, (2) Among current users, what is the frequency of Internet use, types of Internet activities in which they engage, physical and functional difficulties using the Internet, and their comfort level in joining online health discussion groups and exchanging emails with other participants?, (3) Among previous users, what are the reasons for discontinued use?, and (4) Among never users, what is their level of willingness and comfort to engage in conducting health-information searches and joining online health discussion groups and exchanging emails with other participants? In addition, we tested the following hypotheses: Controlling for age, (H1) Never users and previous users will be more likely than current users to be black or Hispanic, lack English proficiency, have lower income, have more chronic medical conditions and ADL/IADL impairments, and have self-reported diagnoses of depression and anxiety, (H2) Among current users, eHealth literacy will be lower among those with self-reported diagnoses of depression and anxiety and lower frequency of Internet use, and (H3) Among never users, attitudes toward computer/Internet use (efficacy and interest) will be positively associated with higher income and living alone and negatively associated with lack of English proficiency and self-reported diagnoses of depression and anxiety.
Participants and Procedures
The data are from a survey of 980 HDM recipients residing in central Texas. The HDM program is operated by a multipurpose aging-service agency that serves about 2100 low-income, disabled, and homebound adults (80% were 60 years and older, and 20% under age 60) daily. The agency receives partial funding for HDM and case management services for clients aged 60 and older under Title III of the Older American Act and partial funding for the same services for persons under age 60 from the state Medicaid program and the Social Services Block Grant. The computer/Internet use survey was conceived as part of the agency’s strategic exploration of the potential for using emails as part of its communication with clients and for providing case management services via videoconferencing in the future. Through an academic and community collaborative partnership, one of the authors helped the agency develop the survey questionnaire and trained the HDM program’s 12 full-time case managers to conduct the survey with their clients. Most surveys were done between November 2012 and February 2013, either in person using the paper form survey questionnaire or by telephone using the electronic form survey questionnaire. The Spanish version of the survey was used for the clients who spoke Spanish only (< 3% of respondents). Clients assessed as having moderate to severe cognitive impairment based on the 4-item (memory, concentration, orientation, and decision-making) cognition test contained in the HDM program’s intake and recertification assessments were excluded from the survey. Clients unable to participate in the survey due to severe mental illness and those who refused to participate for any reason were also excluded. All survey data were entered in the agency’s centralized electronic client data management system and linked with each respondent’s intake or most recent recertification assessment data. With approval from the University of Texas at Austin’s Institutional Review Board, the de-identified data were analyzed in March 2013.
Internet use was measured with the question, “Have you ever used the Internet?” The answer categories were (1) No, I have never used it (never user), (2) I have used it before but not currently (previous user), and (3) Yes, I am a current user. The previous users (n=75 adults under age 60 and n=120 adults aged 60 or older) were asked the reasons for discontinuation (no computer or Internet connection at home because of cost, it is not helpful, I do not need it, cannot use computer because of disability or pain, and other—specify).
Internet Use Patterns and Activities
Patterns and activities in Internet use among current users (n=73 adults under age 60 and n=128 adults aged 60 or older) were ascertained with the following items: (1) location of Internet connection (at home, apartment complex, family/friend’s home, and other—specify) and frequency of Internet use (at least once a day, every few days, once a week, a few times a month, once a month or less often), (2) type of activities conducted on the Internet (research health-related information, research information about other topics or issues of interest, send/receive email, buy products online, do banking online and/or pay bills, read news, papers, magazines, and books online, play games online, watch videos (including YouTube), use social networking or dating site (eg, Facebook, Match.com), and other—specify, (3) ease/difficulty of locating websites the user was looking for and finding the information that he/she needed within that site (on a 5-point Likert scale: 1=“always easy” to 5=“very difficult”), and (4) any physical/functional problem that made it harder for the respondent to use the Internet (pain in the limbs, unsteady hands, difficulty concentrating for long periods of time, difficulty sitting for long periods, eyes that tire easily, and other—specify).
eHealth Literacy among current Internet users was measured by the 8-item eHealth Literacy Scale (eHEALS) with each item scored on a 5-point Likert scale. The eHEALS measures the concept of eHealth literacy as defined as a set of skills required to effectively engage information technology for health and has shown high levels of internal consistency and good test-retest reliability . The items are: (1) I know what health resources are available on the Internet, (2) I know where to find helpful health resources on the Internet, (3) I know how to find helpful health resources on the Internet, (4) I know how to use the Internet to answer my questions about health, (5) I know how to use the health information I find on the Internet to help me, (6) I have the skills I need to evaluate the health resources I find on the Internet, (7) I can tell high-quality health resources from low-quality health resources on the Internet, and (8) I feel confident in using information from the Internet to make health decisions. The final eHEALS score is the average of all 8 items, with higher scores suggesting higher eHealth literacy. The internal consistency reliability in the original eHEALS validation study with a sample of 664 individuals aged 13-21 was Cronbach alpha=.88, and the principal components analysis produced a single factor solution (factor loadings ranging from .60 to .84 among 8 items; eigenvalue=4.479; and 56% of the variance explained). Item-scale correlations ranged from r=.51 to r=.76, and test-retest reliability showed modest stability over a 6-month period (r=.49 to r=.68) . Although the eHEALS was originally validated with adolescents and young adults, it has been used to measure eHealth literacy among older adults . In the present study, internal consistency reliability coefficient for the 8-item eHEALS for current Internet users was Cronbach alpha=.93 for both the younger (under age 60) group (n=73) and the older (60 years and up) group (n=128) adults.
In addition to these 8 items, two eHEALS supplemental items were used to measure (1) perceived usefulness of the Internet in helping make health decisions, and (2) perceived importance of being able to access health resources on the Internet. Both items were scored on a 5-point Likert scale with higher scores suggesting higher levels of perception. eHEALS developers recommend using these two supplemental items along with the 8 items.
Attitudes Toward Computer/Internet
Among never users (n=69 adults under age 60 and n=515 adults aged 60 or older), attitudes toward computers/Internet were measured with the 5-item computer efficacy subscale and the 5-item computer interest subscale of the Attitudes Toward Computers Questionnaire (ATCQ), with each item scored on a 1-5 point Likert scale. The ATCQ was originally validated to measure seven dimensions of attitudes toward computers (comfort, efficacy, gender equality, control, dehumanization, interest, and utility) among 398 students in Grades 4 through 12 . It was later validated with 420 older adults and used to assess older adults’ attitudes toward computers in previous studies [30,37,39,40]. In the present study, the original wording “computer” was changed to “computer/Internet” in each item (ATC/IQ hereafter) to emphasize the Internet. The efficacy items were (1) I know that if I worked hard to learn about computers/Internet, I could do well, (2) Computers/Internet are not too complicated for me to understand (italics in original), (3) I think I am the kind of person who would learn to use a computer/Internet well, (4) I think I am capable of learning to use a computer/Internet, and (5) Given a little time or training, I know I could learn to use a computer/Internet. The interest items were (1) Learning about computers/Internet is a worthwhile and necessary subject, (2) Reading or hearing about computers/Internet would be (is) boring, (3) I don’t care to know more about computers/Internet, (4) Computers/Internet would be (are) fun to use, and (5) Learning about computers/Internet is a waste of time. For both efficacy and interest subscales, the final score is the average of all 5 items, and higher scores suggest higher computer/Internet efficacy or interest. In the present study, the internal consistency reliability coefficients for the efficacy subscale were Cronbach alpha=.85 for those under age 60 and .92 for those age 60 and up. The internal consistency reliability coefficients for the interest subscale were .88 for those under age 60 and .84 for those age 60 and up.
Willingness to Use Online Health Information
Among never users, willingness to use online health information was measured with one item, “If someone can teach me how to use the Internet to look for health information, I am willing to try” and scored on a 1-5 point Likert scale. Higher scores suggest greater willingness.
Comfort With Joining Online Health Discussion Groups and Exchanging Emails With Other Participants
Among both current and never users, comfort with joining online health discussions and exchanging emails with other participants was measured with one item, “I would be comfortable joining an online health discussion group and exchanging emails with other participants” and scored on a 1-5 point Likert scale. Higher scores suggest greater levels of comfort.
Sociodemographics included age in years, gender, race/ethnicity (nonHispanic white—reference category, black/African American, Hispanic, and other), marital status (married, widowed, divorced/separated, and never married), living arrangement (living alone, living with spouse [and any other person], living with another adult, and living with dependent adult child[ren] or minor child[ren] only), income-to-needs ratio (ratio of income to the official poverty line adjusted for the number of family members, with higher ratios indicating higher income/better financial situation), and primary language the respondent speaks at home (English, both English/Spanish, Spanish only). The agency assessment does not include any questions about the client’s level of education.
Health, Mental Health, and Disability
Health, mental health, and disability were measured by (1) the number of chronic medical conditions (arthritis, hypertension, diabetes, heart disease, lung disease, kidney disease, stroke, and cancer) that the respondent reported as having been diagnosed by a doctor, (2) diagnoses of depression, anxiety, and severe mental illnesses that the respondent reported (and confirmed by case managers using the respondent’s list of medications when possible), and (3) the number of ADL impairments (feeding/eating, bathing, grooming, dressing, toileting, and getting in and out of bed) and IADL impairments (cleaning, preparing meals, doing laundry, grocery shopping, making telephone calls, and taking medications).
First, we examined data integrity and missingness using univariate frequency distributions. Of 983 surveys conducted, 3 had incomplete data, leaving 980 usable surveys. Then we examined survey participants’ representativeness by comparing their sociodemographic characteristics to those of all HDM clients the agency serves. Despite the exclusion of cognitively impaired and some severely mentally ill clients, survey participants’ sociodemographic characteristics did not differ statistically from all HDM clients. Next, we conducted bivariate analyses using chi-square and t tests to compare respondents under 60 years of age with respondents 6 years of age and over in terms of their computer/Internet use patterns, reasons for discontinued use, eHealth literacy, attitudes toward computer/Internet use, and their willingness and comfort level regarding joining online health discussion groups and exchanging emails with other participants. Finally, we used multinomial logistic regression models to test H1 (Internet use/previous use/never use) and ordinary least squares (OLS) regression models to test H2 (correlates of eHealth literacy) and H3 (computer/Internet efficacy and interest), with sociodemographic and health/mental health characteristics as covariates. To examine correlates that may be specific to older adults (age 60 and up), we also ran separate multivariate models for them. In the multivariate regression analyses, 12 respondents who were not nonHispanic white, black, or Hispanic were excluded because of their small number. All statistical analyses were conducted with SPSS v.20.
Table 1 shows that the participants consisted of 217 (22.1%) people under age 60 and 763 (77.9%) people aged 60 years or older, 70% were female and 30% male, and 42% were nonHispanic white, 36% black, and 21% Hispanic. (The younger group ranged in age from 30-59 years, with 75% at 50-59. The oldest person in the older age group was 102 years old.) Participants’ median income-to-needs ratio was 1.15, they had an average of three chronic illnesses, 42% reported a depression diagnosis, and 23% reported an anxiety diagnosis. Participants were indeed low-income and had high levels of physical, functional, and mental distress. The two age groups did not differ in terms of gender, racial/ethnic distributions, and number of ADL impairments, while the older age group had greater income and higher numbers of chronic illnesses and IADL impairments but lower rates of self-reported diagnoses of depression, anxiety, and severe mental illness. The majority of both groups lived alone, but their living arrangements and marital status differed (eg, the older group was much more likely to be widowed and the younger group never married).
Internet Use and Correlates
Table 2 shows participants’ Internet use patterns. Almost 60% of all participants had never used the Internet, 20% had used it before, and 20% were currently using it. Internet use did differ by age, with a higher rate of use for younger adults. For example, 38% of those under age 55 and 28% of those aged 55-59 were current users, while less than 15% of those 70 years and older were current users; however, only 10% of those 75-79 years old were current users while close to 15% of those 80-89 years were current users. As expected, those 90 years and older had the smallest proportion of current users (less than 9%). The proportion of previous users also varied by age but with higher rates of discontinued use among younger than older adults. A majority of previous users (76% of the under age 60 group and 60.8% of the 60 and up group) reported their inability to afford an Internet subscription and/or a new computer. Some reported that their old computers were no longer working/broken. In addition, 13.3% of the younger group and 14.2% of the older group reported they stopped using computers/the Internet because of disability, pain, or vision impairment. 9.3% of the younger group and 18.3% of the older group reported they no longer use the Internet because they do not need it. As well, 1.3% of the younger group and 1.7% of the older group reported discontinuing use because the Internet was not helpful. 5% of the older group reported they just moved and needed to reassemble/reconnect their computer before they could use it again, and one person (77 years old) reported that he did not have enough time for the Internet. Age group differences in reasons for discontinuing use were not significant (Pearson χ24=7.91, P=.10)
Table 3 shows the results of multinomial logistic regression analysis. Among all participants, having never used the Internet, as opposed to current use, was significantly associated with older age, being black or Hispanic (as opposed to being nonHispanic white), and having lower income. Odds ratios show that blacks were 4.4 times and Hispanics were 4.7 times more likely than non-Hispanic whites to have never used the Internet (OR 4.41; 95% CI 2.82-6.91, P<.001 for blacks and OR 4.69; 95% CI 2.61-8.44, P<.001 for Hispanics) when other variables were held constant. The likelihood of having never used the Internet decreased by 36% for every one unit increase in income-to-needs ratio (OR 0.36; 95% CI 0.27-0.49, P<.001). Discontinued use, as opposed to current use, was not significantly associated with age; however, it was significantly associated with ethnicity, with blacks almost twice and Hispanics almost three times more likely to have discontinued Internet use (OR 1.79; 95% CI 1.08-2.95, P<.05 for blacks and OR 2.86; 95% CI 2.53-5.35, P<.001 for Hispanics). In addition, the likelihood of discontinued use decreased by 62% with every one unit increase in income-to-needs ratio (OR 0.62; 95% CI 0.45-0.86, P<.001).
Multinomial logistic regression results for older adults only show that the likelihood of having never used the Internet decreased by 62% with a self-reported depression diagnosis (OR 0.62; 95% CI 0.39-0.98, P<.05), suggesting that depressed older adults were more likely to have used the Internet than their peers without this diagnosis. The likelihood of discontinued Internet use decreased by 53% for older adults living alone (OR 0.53; 95% CI 0.31-0.92, P<.05), indicating that older adults living alone are more likely than those living with others to use the Internet.
[view this table]
|Table 1. Sociodemographic and health and mental health characteristics by age group.|
[view this table]
|Table 2. Internet use status by age group (%).|
Internet Use Patterns and Activities Among Current Users
Table 4 shows that 86% of the younger group and 95% of the older group had an Internet connection at home, while the rest used an Internet connection available at their apartment complex, family/friends’ home, and other places including a store with wireless services. The age groups differed significantly in terms of their Internet connection sites (P=.02), with the older group more likely to connect at home; however, they did not differ in frequency of Internet use. A little more than half of both younger and older groups used the Internet daily, a little over 20% used it every few days, 10-11% used it once a week, and the rest used it less often than weekly. For both groups, sending and receiving email was the most popular Internet activity, followed by research on nonhealth- and health-related information. A little more than 75% of the younger group reported health-related information searches, but only 55% of the older group reported the same (P=.01). Significant age group differences were also found in Internet use for playing games (56% of the younger group vs 39% of the older group, P=.03), watching videos (49% vs 27%, P=.01), and for social network or dating sites (48% vs 20%, P<.001). Although not statistically significant, almost half of the younger group engaged in online goods purchases and banking/bill paying and reading online papers/news, magazines and books, while 35-39% of the older group did the same. “Other” Internet use (not reported in Table 4) included collecting coupons, looking at used car/motorcycle pictures on craigslist, looking at other photos, video chats using “Skype and Tango”, checking lottery winning numbers; using it as Yellow pages/directory, listening to the radio, and visiting religious websites. Both age groups reported that they had a relatively easy time finding the information they were looking for on the Internet: mean 2.14 (SD 1.08) for the younger group and mean 2.27 (SD 1.15) for the older group, t=0.82, P=.41), suggesting that they felt confident about their search skills. However, many reported discomfort in using the computer/Internet due to physical, functional, and vision-related limitations. The problems they reported included arthritic pain in the fingers, neck and back pain, neuropathy, difficulty typing due to other disability, chronic fatigue and other medical conditions that interfere with their ability to concentrate and sit for a long period of time, and glaucoma and other vision problems. In addition, a few older adults said they were “too old to learn new things,” were fearful of radiation exposure from the computer, and had insufficient reading comprehension to effectively use Internet resources.
[view this table]
|Table 3. Correlates of Internet use and nonuse: odds ratios (OR) from multinomial regression results.|
[view this table]
|Table 4. Internet use patterns and activities among current users.|
eHealth Literacy, Attitudes Toward Computer/Internet, and Correlates
Table 5 shows eHEALS scores for current users and ATC/IQ scores for never users by age group. eHEALS scores suggest that self-rated eHealth literacy for both age groups, on average, are at a neutral (ie, “undecided”) level, although the younger group’s score was significantly higher than the older group’s: mean 3.53 (SD 0.76) vs mean 3.22 (SD 0.85), t=2.57, P=.01. Both age groups scored slightly higher on the Internet’s usefulness in helping them make decisions about their health (mean 3.67 (SD 1.11) and mean 3.41 (SD 1.28), t=1.48, P=.14) and about the importance of being able to access health resources on the Internet (mean 3.75 (SD 1.16) and mean 3.43 (SD 1.32), t=1.74, P=.08). eHEALS scores and supplemental item scores were highly correlated (r=0.67, P<.001 for eHEALS and perceived usefulness and r=0.69, P<.001 for eHEALS and perceived importance). Perceived usefulness scores and perceived importance scores were also highly correlated (r=0.79, P<.001). However, both groups expressed a lower level of certainty/comfort about joining online health discussion groups and exchanging emails with other participants, with less willingness among older adults: mean 2.89 (SD 1.23) for the younger group and mean 2.52 (SD 1.30) for the older group, t=2.01, P=.05.
Both ATC/IQ efficacy and interest scores, on average, suggest a neutral (undecided) level of efficacy and interest. The younger group’s efficacy score was slightly and significantly higher than the older group’s: mean 3.34 (SD 0.77) vs 3.01 (SD 0.98), t=2.66, P=.01. However, there was no age group difference in interest level: mean 3.33 (SD 0.88) vs 3.23 (SD 0.85), t=0.88, P=.38. Willingness to use online health information (if someone taught them how to use the computer/Internet) was also at a neutral level, with slightly and significantly higher scores for the younger than older group: mean 3.52 (SD 1.05) vs 3.03 (SD 1.22), t=3.18, P=.01. Comfort with joining online health discussion groups and exchanging emails with other participants was also slightly and significantly higher for younger than older adults: mean 2.81 (SD 1.20) vs 2.44 (SD 1.17), t=2.50, P=.01.
Another way to examine eHEALS and ATC/IQ scores is by the proportion of those who agreed (rating of 4) or strongly agreed (rating of 5) with the scale items. Among respondents, 35.7% of the younger group and 29.0% of the older group had an average score of 4+ on the 8-item eHEALS, 65.7% of the younger and 63.3% of the older groups rated perceived usefulness as 4 or 5, 68.5% of the younger and 62.5% of the older groups rated perceived importance 4 or 5, and 39.7% and 31.3% of the respective age groups rated their comfort level with joining online health discussion groups and exchanging emails with other participants as 4 or 5. Further analysis also showed that 40.5% of the younger group and 27.3% of the older group scored 4+ on the ATC/IQ efficacy subscale, 36.2% of the younger group and 28.8% of the older group scored 4+ on the ATC/IQ interest subscale, 68.1% and 48.4% of the respective age groups scored 4 or 5 on their willingness to try accessing online health information, and 40.5% and 25.0% of the respective age groups scored their comfort level with joining online health discussion groups and exchanging emails with other participants as 4 or 5.
As expected, Table 6 shows that eHEALS scores among current Internet users of all ages were inversely associated with age and computer/Internet use frequency, with these variables alone explaining 27% of the variance in eHEALS scores. No other sociodemographic and health/mental health characteristics were associated with eHEALS scores. Among the older age group, however, depression was also inversely associated with eHEALS scores: B=-0.33 (SE 0.14), P=.02. The model adjusted R2 was .36.
As Table 7 shows, ATC/IQ efficacy among never users of all ages was positively associated with living alone, income-to-needs ratio, and the number of medical conditions and inversely associated with age, Hispanic ethnicity, and Spanish as the primary language spoken at home. Among the older age group, being black was also marginally positively associated with higher efficacy scores: B=0.19 (SE 0.10), P=.057. ATC/IQ interest among never users of all ages was positively associated with being black but inversely associated with age, Hispanic ethnicity (marginally), and Spanish as the primary language spoken at home. Among the older age group, Hispanic ethnicity and Spanish as the primary language were not significant factors, while living alone was marginally positively associated with higher interest scores: B=0.14 (SE 0.07), P=.066. Given the low R2 statistics (15% for efficacy and 8-10% for interest), it appears that variables not captured by the participants’ sociodemographic and health/mental health characteristics may influence ATC/IQ efficacy and interest.
[view this table]
|Table 5. eHealth Literacy (eHEALS), Attitudes Toward Computer/Internet (ATC/IQ), and willingness to use health information searches and online health discussion groups: mean and standard deviation of the mean.|
[view this table]
|Table 6. Correlates of eHEALS among current Internet users.|
[view this table]
|Table 7. Correlates of ATC/IQ efficacy and interest among those who never used the Internet.|
Many previous studies describe the digital divide between older and younger people, with the divide being greater for those who are racial/ethnic minorities and of lower socioeconomic status (SES) [19,22,25,28,34]. Much research has also been done regarding the psychological, functional, and educational barriers that prevent many older adults from joining the digital age and taking advantage of the many benefits that Internet technology can offer [18,20,35,36]. Most previous research included older adults with varying degrees of SES and functional abilities, and few focused on those with low-incomes and disabilities. This study examined Internet use patterns, eHealth literacy, and attitudes toward computer/Internet among a large sample of low-income homebound older adults and compared them to a younger group of low-income homebound individuals. Because of their disabilities, none worked for pay and all received home-delivered meals; thus, any computer/Internet use in which they engaged would have been exclusively for their personal use.
As expected, this vulnerable group of individuals had low rates of Internet use—34% of the younger group (under 60 years) and 17% of the older group (60 years and older). For the most part, age was inversely associated with Internet use, with lower use rates among those with more advanced age. However, this study also found that the numbers who had to discontinue Internet use (35% of the younger group and 16% of the older group) were as many as those who were currently using the Internet. The primary reasons for discontinuation were the cost of an Internet connection and/or replacing a nonfunctional computer and disability, pain, and vision problems. For those who could not afford an Internet subscription, computer, or other necessary equipment, it appears that they could have continued using the Internet if affordability was not a barrier.
The study also found that even within this group of low-income homebound persons, racial/ethnic minorities and those with lower income were much less likely to use the Internet, indicating the persistent negative effects of racial/ethnic minority status and low SES on digital inclusion. Multivariate analyses also identified the number of chronic medical conditions and ADL/IADL impairments as marginally significant correlates of Internet use. As expected, ADL/IADL impairments were barriers to Internet use; however, higher numbers of chronic medical conditions were positively associated with current Internet use as opposed to discontinued use. While having chronic medical conditions seems to promote Internet use, it is not clear if this results from the need to obtain medical information or manage ones’ health care or whether having more chronic conditions results in greater isolation, making the Internet important in maintaining social connections. Also important was that a self-reported diagnosis of depression or anxiety was not a significant correlate, suggesting that these mental health conditions did not inhibit Internet use. Thus, H1 was partially supported.
Regardless of age, about three-fourths of Internet users went online either daily or every few days, suggesting that a majority of these low-income, homebound Internet users integrated Internet technology in their usual routines. In this respect, they were not different from the general US population. The aforementioned national survey done in 2012 found that 70% of US adults aged 65 and older used the Internet on a typical day and 67% of all American adults aged 18 and older did so . Younger persons in our sample reported utilizing the Internet for a wider variety of activities, taking advantage of the many benefits the Internet offers. Older users were not taking full advantage of the Internet’s multiple benefits, as only about 55% did research on health-related information and only one-fifth were using the Internet for social networking purposes. This may reflect their reports that searching and finding the information they needed fell somewhere between “sometimes easy” and “not so easy.” Although most users in the older age group used the Internet daily or almost daily, they appear to have somewhat low levels of confidence in their Internet search skills, which was reflected in their average eHEALS score being significantly lower than in the younger age group. In multivariate analyses, eHEALS scores were negatively associated with age, meaning those of older ages had lower perceptions of their eHealth self-efficacy. Better news is that self-efficacy was significantly determined by the frequency of Internet use, a finding consistent with previous studies that training and practicing Internet skills increases older adults’ computer-related self-efficacy [18,29-32]. As hypothesized, eHEALS scores in the older age group were also inversely associated with a depression diagnosis, which may reflect the generally negative self-evaluation among depressed older adults, even though older adults with depression were more likely to use the Internet than those without this diagnosis. These findings support H2. The low-level of eHealth efficacy among both age groups may also have been responsible for respondents’ lack of willingness to join online health discussion groups, although the lack of willingness may also have stemmed from privacy concerns related to the exchange of emails with other participants or warnings to avoid being taken advantage of online. Some low-income homebound older adults may have faced discrimination or mistreatment by others throughout their lives, and as a result, they may have been cautious about connecting with strangers online, although they perceived the usefulness and importance of the Internet-based health resources.
Those who never used the Internet did not express aversion to learning to use it and believed they could do so, although older respondents were slightly less confident than their younger counterparts. In addition to age, Hispanics and those who primarily spoke Spanish at home expressed lower levels of computer/Internet efficacy, though it is not clear why Hispanics, even controlling for Spanish speaking, had a lower level of efficacy. Participants who lived alone and participants, including those in the older age group, with higher incomes had higher efficacy levels. Those living alone may have more confidence in general given their ability to live independently and may also feel a greater need to use email and other Internet technology that can connect them to others.
Although bivariate analyses showed that low-income homebound older adults’ computer/Internet interest did not differ significantly from their younger counterparts, multivariate analyses again showed that older age was a significant negative factor. Although blacks were less likely to use the Internet than nonHispanic whites, black older adults and older adults who lived alone expressed greater interest in computer/Internet use than their nonHispanic white counterparts and those who lived with someone else. Hispanics were also less likely than nonHispanic whites to use the Internet, but ethnicity and Spanish-language use were not significant factors for older adults’ level of interest. Thus, H3 was partially supported.
A few study limitations should be noted. First, despite the large sample size, the survey participants were selected from a geographically limited area, which may limit the findings’ generalizability. Second, although case managers confirmed participants’ reported health and mental health conditions using medication lists, these diagnoses by themselves may not be equated with symptom manifestations and severity, especially in the case of depression, anxiety, and severe mental illness. Pharmacotherapy may have effectively reduced symptoms for some people. Thus, actual assessment of current symptoms would have allowed more accurate evaluation of their influence. Third, a potentially important omission in the agency assessment dataset was the clients’ level of education, which could have provided a more comprehensive picture of participants’ sociodemographic characteristics and their level of disadvantage.
Despite its limitations, the present study is the first to provide a detailed description of Internet use and nonuse among low-income, disabled, and homebound older and younger adults. We found very low rates of Internet use compared to the US population, either due to lack of exposure to computer and Internet technology, lack of financial resources to obtain computers and access technology for personal use, or medical conditions, disabilities, and associated pain that restrict use. However, the findings also provide hope for reducing the digital divide because while blacks and Hispanics were less likely to be current Internet users, black older adults expressed greater interest and Spanish-speaking adults were no less interested than nonHispanic whites in computer/Internet use. Internet technology can offer multiple benefits to vulnerable homebound individuals and may contribute to reducing health disparities among other disabled homebound individuals. Thus, improving access to Internet technology that can enhance health and well-being should be viewed as a social justice issue. As health care systems increasingly rely on Internet technology to manage patients’ records, communicate with them, and provide care, it will be necessary for patients who want greater involvement in their health care to become proficient in using health information technology. For example, access to one’s electronic health records and being able to communicate more readily with health care providers may go a long way in improving patient compliance with treatment regimens, engaging patients in their treatment, and increasing their control over what is happening to them. Those, such as homebound individuals who often have multiple health needs, can benefit the most from becoming well versed at using these tools.
Until the digital divide eventually disappears due to younger Americans’ nearly universal exposure to computer use and Internet technology, steps can be taken to increase Internet use among today’s older adults, including those who are disadvantaged, through social policy, technology/equipment design, and training/education. First, in the social policy domain, offering low-income persons technology subsidies/allowances may help them join the digital age, and government agencies and nongovernmental organizations (NGOs) could recycle and refurbish the many computers sitting idle or disposed of each year for older adults’ use. Such government-provided allowances and NGO efforts may result in cost effectiveness if they enable people to live independently, reduce their dependence on informal and formal support, and increase their quality of life. Second, to encourage Internet use among individuals with substantial disabilities, technology should be designed to be as user-friendly as possible . For example, touch screens tend to be more usable than keyboards for older adults with arthritic pains in their fingers/hands. However, future technology designs should go far beyond many current innovations and provide personalized technology systems to facilitate use even among those with a high degree of disability, such as greater access to voice systems for people with low literacy levels or who are visually impaired. Third, older adults who remain reluctant to use the Internet due to low technology-related self-efficacy may be motivated to embrace its use through demonstrations and education. The present study suggests that exposure and practice (ie, frequent use) increases Internet skill efficacy regardless of income level and disability. Many younger people’s computer and Internet skills could be used by employing them through volunteer work or paid employment to teach older adults how to use email, surf the Web, and engage in social networking and health-related tasks. Especially for low-income homebound older adults who have not been exposed to computer/Internet technology but whose needs are substantial, the multiple benefits of computer/Internet use need to be emphasized and equipment and training provided to facilitate their use.
Future studies should test the extent to which providing computer equipment (especially devices that are easiest to use), Internet connections, computer applications, and training to use them increase interest, use, and efficacy and are associated with improved health, mental health, and other well-being outcomes. Studies should also test the types of devices and applications that homebound adults may be most interested in using and the features that facilitate use and better outcomes for this population.
The authors are grateful to the HDM program’s case managers for data collection, to Linda Perez for her supervision of data collection, and to David Thompson for his data management.
Conflicts of Interest
- United States Census Bureau. Statistical Table Based on data from the 2009 American Community Survey. 2009. Table 35, Persons 65 Years and Over - Living Arrangements and Disability Status URL: http://www.census.gov/compendia/statab/2012/tables/12s0035.pdf [accessed 2013-04-01] [WebCite Cache]
- Centers for Medicare and Medicaid Services. Table 7.2, Persons Served, Visits, Total Charges, Visit Charges, and Program Payments for Medicare Home Health Agency Services: Calendar Year 2011. 2012. Medicare and Medicaid Supplemental Statistics URL: http://cms.hhs.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/MedicareMedicaidStatSupp/2012.html [accessed 2013-04-01] [WebCite Cache]
- Barrett A, Schimmel J. Issue Brief No 2.: Mathematica Policy Research, Inc; 2012 Aug. OAA Title III services target the most vulnerable elderly in the United States URL: http://www.nanasp.org/pdf/vulnerableseniors_ib2.pdf [accessed 2013-04-24] [WebCite Cache]
- Altchuler N, Schimmel J. Issue Brief No. 1.: Mathematica Policy Research, Inc; 2010. Aging in place: Do Older Americans Act Title III services reach those most likely to enter nursing homes? URL: http://www.aoa.gov/aoaroot/program_results/docs/AoA-issue1_Nursing%20Homes.pdf [accessed 2013-04-24] [WebCite Cache]
- Colello KJ. Congressional Research Service Report for Congress. 2011 Jun 17. Older Americans Act Title III Nutrition Services Program URL: http://www.aging.senate.gov/crs/nutrition1.pdf [accessed 2013-04-24] [WebCite Cache]
- Choi NG, Teeters M, Perez L, Farar B, Thompson D. Severity and correlates of depressive symptoms among recipients of meals on wheels: age, gender, and racial/ethnic difference. Aging Ment Health 2010 Mar;14(2):145-154. [CrossRef] [Medline]
- Ell K, Unützer J, Aranda M, Sanchez K, Lee PJ. Routine PHQ-9 depression screening in home health care: depression, prevalence, clinical and treatment characteristics and screening implementation. Home Health Care Serv Q 2005;24(4):1-19 [FREE Full text] [CrossRef] [Medline]
- Sirey JA, Bruce ML, Carpenter M, Booker D, Reid MC, Newell KA, et al. Depressive symptoms and suicidal ideation among older adults receiving home delivered meals. Int J Geriatr Psychiatry 2008 Dec;23(12):1306-1311. [CrossRef] [Medline]
- Grady B, Myers KM, Nelson EL, Belz N, Bennett L, Carnahan L, American Telemedicine Association Telemental Health Standards and Guidelines Working Group. Evidence-based practice for telemental health. Telemed J E Health 2011 Mar;17(2):131-148. [CrossRef] [Medline]
- van den Berg N, Schumann M, Kraft K, Hoffmann W. Telemedicine and telecare for older patients--a systematic review. Maturitas 2012 Oct;73(2):94-114. [CrossRef] [Medline]
- Wilcox ME, Adhikari NK. The effect of telemedicine in critically ill patients: systematic review and meta-analysis. Crit Care 2012 Jul 18;16(4):R127 [FREE Full text] [CrossRef] [Medline]
- Internet use among midlife and older adults: an AARP bulletin Poll.: AARP; 2009 Dec. URL: http://assets.aarp.org/rgcenter/general/bulletin_internet_09.pdf [accessed 2013-04-24] [WebCite Cache]
- Crabb RM, Rafie S, Weingardt KR. Health-related internet use in older primary care patients. Gerontology 2012;58(2):164-170. [CrossRef] [Medline]
- Czaja SJ, Lee CC. The impact of aging on access to technology. Univ Access Inf Soc 2006 Dec 2006;5(4):341-349. [CrossRef]
- Gatto SL, Tak SH. Computer, Internet, and E-mail Use Among Older Adults: Benefits and Barriers. Educational Gerontology 2008 Aug 2008;34(9):800-811. [CrossRef]
- Russell C, Campbell A, Hughes I. Ageing, social capital and the Internet: findings from an exploratory study of Australian 'silver surfers'. Australas J Ageing 2008 Jun;27(2):78-82. [CrossRef] [Medline]
- Sum S, Mathews RM, Hughes I. Participation of older adults in cyberspace: how Australian older adults use the Internet. Australas J Ageing 2009 Dec;28(4):189-193. [CrossRef] [Medline]
- Xie B. Effects of an eHealth literacy intervention for older adults. J Med Internet Res 2011 Nov;13(4):e90 [FREE Full text] [CrossRef] [Medline]
- Zickuhr K, Madden M. Pew Research Center's Internet & American Life Project. Older adults and Internet use URL: http://www.pewinternet.org/Reports/2012/Older-adults-and-internet-use.aspx [accessed 2013-04-24] [WebCite Cache]
- Adams N, Stubbs D, Woods V. Psychological barriers to Internet usage among older adults in the UK. Med Inform Internet Med 2005 Mar;30(1):3-17. [CrossRef] [Medline]
- Carpenter BD, Buday S. Computer use among older adults in a naturally occurring retirement community. Computers in Human Behavior 2007 Nov;23(6):3012-3024. [CrossRef]
- Choi N. Relationship between health service use and health information technology use among older adults: analysis of the US National Health Interview Survey. J Med Internet Res 2011;13(2):e33 [FREE Full text] [CrossRef] [Medline]
- Dreisbach N, Axler F, Kotranski L, Weingartner RM, Ingerman S, Ramirez N, et al. Assessing the digital divide among older adults residing in an urban and suburban metropolitan area. In: Proceedings of the 139th Annual Meeting of the American Public Health Association. 2011 Presented at: 139th Annual Meeting of the American Public Health Association; Oct. 29 - Nov. 2, 2011; Washington, DC URL: http://www.chdbdata.org/uploads/datareports/Internet&Older%20Adults_10.28.pdf [WebCite Cache]
- Slegers K, van Boxtel M, Jolles J. Computer use in older adults: determinants and the relationship with cognitive change over a 6 year episode. Comput Hum Behav 2012;28:1-10. [CrossRef]
- Werner JM, Carlson M, Jordan-Marsh M, Clark F. Predictors of computer use in community-dwelling, ethnically diverse older adults. Hum Factors 2011 Oct;53(5):431-447 [FREE Full text] [Medline]
- Charness N, Boot WR. Aging and information technology use. Curr Dir Psychol Sci 2009;18(5):253-258. [CrossRef]
- Choi NG, DiNitto DM. Different types of Internet use among older adults:. Association with health needs, psychological capital, and social capital 2013 (forthcoming). [CrossRef]
- Berry R. Older people and the Internet. UK: International Longevity Centre; 2011. URL: http://www.ilcuk.org.uk/images/uploads/publication-pdfs/pdf_pdf_181.pdf [accessed 2013-04-24] [WebCite Cache]
- Chu A, Huber J, Mastel-Smith B, Cesario S. "Partnering with Seniors for Better Health": computer use and Internet health information retrieval among older adults in a low socioeconomic community. J Med Libr Assoc 2009 Jan;97(1):12-20 [FREE Full text] [CrossRef] [Medline]
- Czaja SJ, Sharit J. Age differences in attitudes toward computers. J Gerontol B Psychol Sci Soc Sci 1998 Sep;53(5):P329-P340. [Medline]
- Wild KV, Mattek NC, Maxwell SA, Dodge HH, Jimison HB, Kaye JA. Computer-related self-efficacy and anxiety in older adults with and without mild cognitive impairment. Alzheimers Dement 2012 Nov;8(6):544-552. [CrossRef] [Medline]
- Lagana L. Enhancing the Attitudes and Self-Efficacy of Older Adults Toward Computers and the Internet: Results of a Pilot Study. Educ Gerontol 2008 Sep 1;34(9):831 [FREE Full text] [CrossRef] [Medline]
- Norman CD, Skinner HA. eHEALS: The eHealth Literacy Scale. J Med Internet Res 2006 Nov;8(4):e27 [FREE Full text] [CrossRef] [Medline]
- Czaja SJ, Sharit J, Lee CC, Nair SN, Hernández MA, Arana N, et al. Factors influencing use of an e-health website in a community sample of older adults. J Am Med Inform Assoc 2013 Jul;20(2):277-284. [CrossRef] [Medline]
- Jensen JD, King AJ, Davis LA, Guntzviller LM. Utilization of internet technology by low-income adults: the role of health literacy, health numeracy, and computer assistance. J Aging Health 2010 Sep;22(6):804-826. [CrossRef] [Medline]
- Neter E, Brainin E. eHealth literacy: extending the digital divide to the realm of health information. J Med Internet Res 2012 Jan;14(1):e19 [FREE Full text] [CrossRef] [Medline]
- Xie B. Older adults, e-health literacy, and collaborative learning: An experimental study. J Am Soc Inf Sci 2011 Mar;62(5):933-946. [CrossRef]
- Bear GG, Richards HC, Lancaster P. Attitudes Toward Computers: Validation of a Computer Attitudes Scale. Journal of Educational Computing Research 1995 Jan 1995;3(2):207-218. [CrossRef]
- Jay GM, Willis SL. Influence of direct computer experience on older adults' attitudes toward computers. J Gerontol 1992 Jul;47(4):P250-P257. [Medline]
- Xie B, Bugg JM. Public library computer training for older adults to access high-quality Internet health information. Libr Inf Sci Res 2009 Sep 1;31(3):155 [FREE Full text] [CrossRef] [Medline]
- Wagner N, Hassanein K, Head M. Computer use by older adults: A multi-disciplinary review. Comput Hum Behav 2010;26:870-882. [CrossRef]
|ADL/IADL: activities of daily living and instrumental activities of daily living|
|ATC/IQ: Attitudes Toward Computer/Internet Questionnaire|
|eHEALS: eHealth Literacy Scale|
|HDM: home-delivered meals|
|NGOs: nongovernmental organizations|
|SES: socioeconomic status|
|Edited by G Eysenbach; submitted 02.04.13; peer-reviewed by B Tennant, L Donelle; comments to author 19.04.13; revised version received 21.04.13; accepted 22.04.13; published 02.05.13|
Please cite as:
Choi NG, DiNitto DM
The Digital Divide Among Low-Income Homebound Older Adults: Internet Use Patterns, eHealth Literacy, and Attitudes Toward Computer/Internet Use
J Med Internet Res 2013;15(5):e93
END, compatible with Endnote
BibTeX, compatible with BibDesk, LaTeX
RIS, compatible with RefMan, Procite, Endnote, RefWorks
Add this article to your Mendeley library
Add this article to your CiteULike library
Copyright©Namkee G. Choi, Diana M DiNitto. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 02.05.2013.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.