Telemedicine and electronic health (eHealth) transcends geographic, social, and political boundaries, making them essential tools to leverage health care delivery and surveillance [1,2]. The Internet has become deeply penetrated into society, with nearly 90% of adults in the United States having Internet access [3]. Millennials (18-35-year-olds), however, continue to adopt the Internet at a more rapid rate than members of Generation X (36-51-year-olds), Baby Boomers (52-70-year-olds), and the Silent Generation (71-115-year-olds) [4]. Although age-related disparities in Internet adoption have declined in recent years [5], the strategies to narrow this chasm and optimize the eHealth experience will require a closer look at the unique attributes of generations.

Generational differences in technological adoption can be broadly attributed to the point in one’s life that technology was penetrated into society [6]. Members of Generation X created the same technology that has become central to Millennials’ everyday lives. Rather than being familiar with and growing up with technology, Baby Boomers and members of the Silent Generation were introduced to technology after their social and cultural identities had been established. Widespread adoption of the Internet and the capabilities of technology have led Baby Boomers and members of the Silent Generation, who are traditionally considered late adopters of innovations like technology [4,6], to become excited and willing to adapt and learn about new technologies [7]. However, barriers related to unfamiliarity and uncertainty surrounding the use, value, and security of health information technologies persist among middle-to-older age adults [8-11], especially among those who are not avid health service users [12]. Evidence also shows that non-primary care physicians who are 55 years old and over are less likely to integrate electronic health record systems into their practice, as compared to their younger physician counterparts [13]. Consistent with theoretical underpinnings of the Diffusion of Innovation and the Technology Acceptance Model [7,14], technology tends to be adopted more quickly among younger age groups who find that it is both useful and easy to use.

Adoption of eHealth, however, does not ensure that the technology is used appropriately or that it is used to access high-quality and actionable health information [15]. eHealth literacy, driven by health and computer literacies, is defined as the capacity to locate, understand, evaluate, and act upon health information from technology [16]. People with a low degree of eHealth literacy are less likely to find the Internet as a useful health information tool, to trust the health information from diverse online sources and channels [17], and to actively seek out health information from the Internet [18]. Literacy in eHealth is a central skill set that influences not only health information seeking behaviors [19-21], but also the likelihood of engaging in proactive health-related outcomes and experiences [18,22]. Similar to generational values, researchers argue that social and cultural contextual frames influence eHealth literacy [23,24]. As such, understanding how generational age serves as a function of eHealth literacy and optimizing its measurement across these groups will be critical in the evolving technological era.

Empirical evidence over the past decade has shown that an inverse correlation exists between age and eHealth literacy [18,22,25,26]. Older adults generally have lower health literacy than their younger counterparts [27,28], yet this population is increasingly adopting the Internet with a high degree of confidence to access health information and supplement their health care [4,25]. Paige and colleagues [17] found that adults in the middle-to-older adult age groups, or Boomers and members of the Silent Generation, were more likely to have low eHealth literacy than their younger counterparts. Older age groups were also less likely to trust health information from social support forums but more likely to trust health information from Facebook. These age disparities have been attributed to older adults’ unique health needs as compared to younger adults [29,30], including specialized health information related to chronic disease [31-33], the potential risk for social isolation [34,35], and physical and cognitive limitations that are due to the natural aging process [36]. The generational differences in information seeking behaviors in the non-health context have also been highlighted in the literature to show that Millennials and Baby Boomers consult different informational sources [37]. For these reasons, it would be naïve to assume that eHealth literacy is measured and conceptualized equivalently across generational age groups. To our knowledge, evidence to support that measurement invariance of eHealth literacy scores exists across generations does not exist.

Valid age-group comparisons of eHealth literacy and associated patient-reported outcomes cannot be established without evidence that eHealth literacy measures function equally, or invariant, among young and old adults [38,39]. Measurement invariance indicates that the latent construct captured by an instrument will function similarly across different groups. Multi-group comparisons that do not meet the assumption of measurement invariance are ambiguous and subject to bias [38,40] and may use misleading or false claims to advance research and practice [41]. Without such evidence, it is unknown if the different relationship between eHealth literacy and age is due to real differences or systematic biases. As such, older adults may have a lesser degree of confidence to use eHealth. However, it is also possible that normal age-related cognitive declines [42] and low health literacy [28] readily reported among older adults contribute to depleted attention and working memory to recall accurate responses. As such, specific items may appear more salient in one age group over another. Establishing measurement invariance across eHealth literacy scales will have significant implications for fair and equitable testing standards. Also, it will alleviate bias in using these instruments to identify patients who are likely to benefit from online programs.

Since the conceptualization of eHealth literacy in 2006, several instruments have been developed to capture this construct in the evolving era of eHealth. The seminal instrument, the eHealth Literacy Scale (eHEALS), is a brief 8-item measure with theoretical underpinnings in self-efficacy, or the confidence in one’s capability to engage in behavior to result in the desired outcome [43]. Alongside the emergence of online social environments, like social media, there have been criticisms that the instrument has a compromised degree of content validity [16], particularly regarding its insufficient ability to capture the multidimensional and dynamic features of eHealth. In response, formative research was conducted to derive constructs salient to eHealth literacy and its measurement inductively. The most significant contribution noted by these instruments is the ability to capture the dynamic feature of eHealth and pressing issues related to eHealth use (eg, privacy). These instruments included items that assess if an Internet user can talk to their offline health care provider about the health information found on the Internet [44], as well as their skills related to privacy protection and message self-creation with a keyboard [45]. Another instrument related to eHealth literacy assessed the Internet users preferred mode of interaction and online experiences, as well as their degree of computer anxiety and health information needs [46]. These new instruments tap into unique aspects of eHealth literacy, but they do not provide insight into the communication exchange processes that are missing from eHEALS. Instead, these instruments have been said to leave eHealth literacy literature static, as recent attempts to advance the concept and measurement have not built upon previous literature [47]. Given this information, it is possible that new operational definitions, concepts, and measures that do not build upon seminal work of eHealth literacy may lead researchers astray from the core operational behaviors (ie, locate, understand, evaluate, act upon). Although measures of eHealth literacy have been published, eHEALS remains the most widely used and refined instrument in the literature [48-50] .

Evidence for the internal structure and external validity of eHEALS as a unidimensional measure exists across diverse age groups. These populations include adolescents [51], college students [52], the general adult population [52], patients with chronic disease [53], older adults recruited to surveys conducted online [54], and baby boomers and older adults recruited through the telephone [55]. More recently, studies applying sophisticated psychometric modeling techniques have found that eHEALS is a multi-dimensional measure that captures operational behaviors consistent with the seminal operational definition of eHealth literacy [16]. The eHEALS has been identified as a 2-factor measure of eHealth literacy among Australian adults who are at-risk for cardiovascular disease [56]. The 2-factor model was replicated among general adult populations in Germany [57] and Israel [57,58]. These factors have been defined as measuring information seeking and information appraisal. Most recently, the eHEALS 3-factor structure has been reported among adults later in the lifespan. Sudbury-Riley and colleagues [49] report that eHEALS scores produce a 3-factor model of information awareness, seeking, and appraisal skills among baby boomers. Similarly, the 3-factor model of eHEALS has been confirmed among baby boomers and older adults [55], chronic disease patients [53], as well as middle age adults, with an average age of 53, in a magnetic resonance imaging and computed tomography medical imaging outpatient clinic [59]. The 3-factor structure of eHEALS scores, however, has not been reported or confirmed among younger age groups, like Millennials or members of Generation X. As such, the discrepancy in 1-, 2-, and 3-factors captured among middle-to-older age adults and the general population in international contexts brings into question whether or not eHEALS produces similar factor structures across the lifespan.

Sudbury-Riley and colleagues [49] found measurement invariance for the 3-factor structure among baby boomers in the US, United Kingdom, and New Zealand. As such, the multidimensional eHEALS structure does not vary among Baby Boomers across international borders, regardless of the various health care provisions and coordination that drive social and cultural frames. Age, however, is also a strong determinant that shapes and influences the social and cultural frame of a given population [60]. Baby Boomers are a single generation within the lifespan, and measuring their health-related technological skills is well justified. However, Baby Boomers are a single generation whose socio-cultural and political frame has a significant influence on health outcomes and health services uptake [61-63]. Evidence that research on generational differences in eHealth adoption sets a precedent to also consider the potential generational variability in measures that assess eHealth literacy.

Although a “gold standard” eHealth literacy instrument does not exist [47], the eHEALS remains the closest to reaching this status due to its brevity, popularity, and theoretical underpinnings in health behavior change theory. Researchers have recommended the refinement of the eHEALS, specifically to account for the social nature of eHealth [47,64,65]. Before embarking on this mission, there is an obligation to understand the multidimensional factor structure of the eHEALS across age groups and whether or not these factor structures are invariant. Without such evidence, it will be challenging to refine eHEALS as a reliable measure that produces scores with a high degree of validity evidence across the lifespan in the social era of eHealth. Therefore, the purpose of this study is to examine the structure of eHEALS scores and the degree of measurement invariance among three generations: Millennials, Generation X, and Baby Boomers and the Silent Generation.

A sample of Qualtrics Panelists from the United States completed an online survey in May 2015, which was approved by the university Institutional Review Board (IRB). The sample was stratified by race (ie, Caucasian, Black/African American). Per Qualtrics Panels, the survey functioned as opt-in, meaning that panelists who met inclusion criteria were offered the opportunity to consent to participate. For this particular survey, the inclusion criteria included residing within the United States and being older than 18 years old. Some respondents (n=11) did not provide their age and were subsequently removed from the final sample (N=829). Upon removing respondents who did not provide an age value, there were no missing eHEALS data in this sample.

The following sociodemographic factors were measured across the sample [66]: (1) age (in years), (2) gender (Male, Female), (3) race (Black/African American, Caucasian), (4) ethnicity (Hispanic, non-Hispanic), (5) education level, (6) annual income, and (7) Internet use for Health. The eHealth Literacy was assessed with the eHEALS (Norman and Skinner, [51]), an 8-item, 5-point Likert-type rating scale (1=strongly disagree, 5=strongly agree).

The age group of the sample was categories as: (1) Millennials (18-35-year-olds), (2) Generation X (36-51-year-olds), and (3) Baby Boomers (52-70-year-olds) and Silent Generation (71-115-year-olds) [67]. Baby Boomers and the Silent Generation were collapsed into a single group for this study because the sample only contained 45 members of the Silent Generation. Frequency and descriptive statistics were computed to describe the sample and eHealth literacy scores. A series of chi-squared analyses were conducted to determine if sociodemographic factors were significantly different by age group.

As shown in Table 1, the mean ages of Millennials, Generation X, and Baby Boomers/Silent Generation Members were 26.64 (SD 5.14), 42.97 (SD 5.01), and 62.80 (SD 6.66), respectively. Respondents were mostly female (603/829, 72.74%), earning at least US $35,000 each year (499/829, 60.41%), and living with at least some college experience (623/829, 75.15%%). There were no statistically significant differences in gender, income, or education across each generation group. Nearly half of the respondents were Black/African American (412/829, 49.70%) or Caucasian (417/829, 50.30%), and most were non-Hispanic or Latino (807/829, 97.2%). A greater number of Millennials used the Internet for health-related purposes, as compared with members of Generation X or Baby Boomers/Silent Generation groups (P=.009).

The estimates of model fit for 1-4 factor models are presented in Table 2. Exceeding the acceptable level of RMSEA were the 1-factor (value=.14), 2-factor (value=.09) models. The 3-factor model (RMSEA=.06, 90% CI 0.04-0.08, CFI=.98, TLI=.98) and 4-factor model (RMSEA=.06, 90% CI 0.04-0.08, CFI=.99, TLI=.99) indicated good global model fit. Similarly, the AIC values for the 3-factor (value=12750) and 4-factor (value=12737.50) models were lower than the values for the 1-factor, 2-factor, and 4-factor models.

Although the 4-factor model yielded the best fitting model, items from the scale did not statistically significantly load onto the fourth factor. Therefore, the 3-factor model was used as the basis for assessing measurement invariance among young and old respondents.

Table 3 shows the statistically significant unstandardized factor loadings for the 3-factor model among Millennials, Generation X, and Baby Boomers/Silent Generation groups. Factor 1, which includes items that assess awareness about what health information is available on the Internet and where it can be located, contained significant factor loadings for Items 1-2 across all groups. Similarly, items 5-8 yielded high (greater than .40) and significant loadings on Factor 3, which included items that assess confidence in evaluating and using health information to answer health-related questions. Items 3 and 4, which assessed knowledge about how to use and find helpful health resource on the Internet, had a moderate to strong relationship with Factor 2 across all generation groups. All 3 factors were statistically significantly correlated with one another across all 3 groups. Interestingly, the correlation of Factor 1 with Factors 2 (r=.98) and 3 (r=.80) were much stronger than for the other generation groups. The final 3-factor model used to guide measurement invariance testing is shown in Figure 1.

Table 4 shows the results for configural, pattern, and unique factor invariance tests through the use of a CFA. The 3-factor model has slightly poorer, but acceptable, model fit in regards to configural invariance. This is determined from the RMSEA (value=.08, 90% CI 0.06-0.09) and CFI/TLI (.98 and .97, respectively), confirming that the 3-factor model represents the factor structure of eHEALS across all generations. Adding constraints on the factor loadings across generation groups (pattern invariance testing) resulted in slight improvement of RMSEA and relatively steady SRMR, CFI, and TLI values. The change in chi-square was not statistically significant and the CFI did not deviate by .01. In regard to unique factor invariance, the change in chi-square (Δχ²=69.51, Δdf=16) was statistically significant at. P<.05. As such, unique factor invariance was rejected as equating the error variances of each item across groups significantly diminished the model fit. Moreover, the AIC value for the pattern invariance model (value=12,770.60) was lower than the models testing for configural (value=12,775.72) and unique factor (value=12,808.11) invariance. Therefore, measurement invariance of for the proposed 3-factor structure exists among Millennials, the Generation X, and Baby Boomers/Silent Generation.

Table 5 shows the average scale scores for the 3-factor eHEALS model across each generation. Internal consistency alpha estimates were within appropriate range for each factor, and omega coefficients demonstrated equivalent values to support reliability of the data. A one-way ANOVA showed that eHEALS scores varied across generations for Factor 1 (F [2, 827]=8.17, P<.001), Factor 2 (F [2, 826]=6.00, P=.003), and Factor 2 (F [2, 827]=18.51, P<.001). Tukey honest significant difference (HSD) post hoc analyses showed that, on average, members of the Baby Boomer and Silent Generation groups reported less knowledge and confidence in their eHealth literacy across all factors (P<.05), as compared to members of the Millennial and Generation X groups.

This study examined the degree of measurement invariance in eHEALS scores in the United States belonging to the Millennial, Generation X, and Baby Boomers/Silent Generations. The eHEALS is a multidimensional measure that can be used to assess eHealth literacy across the lifespan consistently. Millennials are more knowledgeable and confident in their online health information awareness, information seeking skills, and information engagement abilities, as compared to members of Generation X and the Baby Boomers/Silent Generation. Further, this study offers significant implications for the continued use and potential refinement of eHEALS in future research and practice-based settings.

The eHEALS scores best fit a positively correlated 3-factor model that captures the following underlying factors: information awareness, information seeking, and information engagement. This finding comes at a time when there is inconsistent evidence for the factor structure of eHEALS. Results of our study contrast with those described by Nguyen and colleagues [52], who explored the dimensionality of eHEALS when it was administered online to a significant proportion (60%) Millennials. Data from Nguyen and colleagues [52] showed eHEALS to have a unidimensional structure with a principal components analysis, which traditionally identifies the fewest number of factors that explain the substantial amount of variance in observed variables [78]. Considering the conflicting evidence describing the dimensionality of eHealth literacy, our alternative multi-group exploratory structural equation modeling approach sought to validate constructs implicit within eHEALS items across three different age groups. Moreover, the current study strived to cast a broader net to explore not only which eHEALS items best explained retained factors, but also how these factors might function in a theoretically driven manner consistent with eHealth literacy literature. Contrary to findings reported by Nguyen and colleagues [52], evidence generated in this study supported a 3-factor model of the English-version of eHEALS. Of note, our sample of Web-based panelists included proportionately more adults representing older Baby Boomer and Silent Generations. These 2 generations were underrepresented in their analyses conducted using an Internet-based sample obtained through machine learning software.

The 3-factor eHEALS model supported in this study captures a more precise assessment of eHealth literacy that goes beyond individual knowledge and perceptions of behavioral capability. The 3-factor eHEALS model comprises items that measure self-efficacy towards central operational skills related to eHealth literacy (ie, locate, evaluate, apply). These operational skills are associated with unique, albeit related, dimensions of self-efficacy in the context of eHealth literacy [16], which explains the highly correlated 3-factor model containing unique factors.

Configural and pattern invariance was upheld across all generation groups in the 3-factor eHEALS model, suggesting that eHEALS scores from the 3-factor model can be interpreted equivalently, regardless of respondents’ age group membership. Despite invariance between groups in the current study, the items that comprise each of these factors are inconsistent with the results of previous literature. Stellefson and colleagues [55] examined the factor structure of eHEALS scores among Baby Boomers and Silent Generation members during a telephone interview and found that Item 3 (ie, “I have the skills I need to evaluate the health resources I find on the Internet”) significantly loaded on both Factor 2 and Factor 3. In that study, Factor 2 included items related to knowing how to use and find helpful health information on the Internet to make informed health decisions. Factor 3 included only 1 other item with a significant factor loading, which addressed the ability to evaluate the quality of online health information. Findings from Stellefson and colleagues [55] run contrary to the current study and also findings reported by Sudbury-Riley and colleagues’ [49], who speculated that the content and theoretical underpinnings of this particular eHEALS item (Item 3) denote skills related to confidence in the ability to evaluate and act upon health information from the Internet. After a closer inspection of Item 3 content, it appears that this question may assess two distinct skills: (1) can one evaluate health information from the Internet? and (2) can one find health information on the Internet? The mode of data collection in Stellefson and colleagues’ [55] study was over the telephone, whereas the data collected in the current study and Sudbury-Riley and colleagues’ [49] was through a Web-based survey. It is possible that respondents only cognitively processed a single operational behavior outlined in this item (ie, find, evaluate), or perhaps the telephone interviewer placed emphasis on one skill over the other. Future research is needed to understand how data collection modality (eg, telephone, online) might directly affect the interpretation of the eHEALS items and ultimately the construct validity of the data produced.

Lastly, the final test of measurement invariance proved to be insufficient. The residual error variances of items were significantly different across age groups when tested within the 3-factor eHEALS model. Unique factor invariance is the strictest form of measurement equivalence. It is rarely achieved in practice, and experts have recently acknowledged that establishing unique factor invariance can be somewhat unreasonable for subjective measurement [79]. Therefore, we suggest that scores produced by the eHEALS may still be used as a comparative index to examine eHealth literacy across age groups [72].

This study sampled opt-in respondents from a Qualtrics Survey Panel taken from the general US population. Despite the population from which the sample was derived, the respondents were predominantly female with a normally distributed income and educational level. Moreover, half of the sample identified as Caucasian and the other half as Black/African American. In other words, this study enrolled over 400 respondents from population subgroups (ie, middle-older age adults, Black/African Americans) that are traditionally underserved in health promotion research. Although this represents a limitation affecting the generalizability of data to the entire US population, the diversity of sample characteristics remains a significant strength of this measurement study.

This was a self-reported Web-based survey, and, therefore, the results of this study can only speak to the interpretation and measurement invariance of scores from eHEALS administered on the Web. There is sufficient reliability and validity evidence of eHEALS delivered via telephone among middle-to-older age adults [55], a population most likely to respond differently to Web-based versus telephone-administered surveys [80]. Future research could explore the degree of measurement invariance of the 3-factor eHEALS model across generations according to the mode of survey administration. Moreover, respondents of this Web-based survey were members of Qualtrics Panels who opted-in to participate. The purposefully racially stratified sample and normally distributed income levels compromises the generalizability of the findings. However, the oversampling of minorities and low-income adults engaged these particularly vulnerable and hard-to-reach populations in survey research.

Although this study did not consider the geographic region (ie, rural versus urban) of the sample, nearly 70% of the sample reported using social media for health-related purposes, which requires a sufficient level of broadband. Rural adults are generally older [81] and have limited broadband connections [82] that enable sustained access to eHealth services. Moreover, rural residents are nearly twice as likely to not use the Internet as compared to their urban counterparts [82,83]. Therefore, factors beyond geographic location may limit rural adults’ eHealth use. Based on the limited empirical evidence related to the eHealth literacy of rural populations [84], future research is needed to explore eHealth literacy and its measurement among populations according to rurality regarding physical space (ie, Rural-Urban Commuting Area or Metropolitan Statistical Area data) and sociocultural rural identity.

Acknowledging the multidimensionality of scores obtained from eHEALS will allow practitioners to obtain a more precise understanding of consumers strengths and weaknesses using the Internet for health-related purposes. Rather than ambiguously interpreting “low eHealth literacy” based on prior unidimensional assumptions underlying eHEALS, practitioners considering the 3-factor model of eHEALS can identify the degree to which their patients have confidence in online health information awareness, search, and engagement. Interpreting scores based on 3 underlying eHEALS dimensions can assist practitioners and researchers to more efficiently direct patients to eHealth resources that are appropriate to their relative skill set, whether it is simply increasing awareness of existing online health information resources or providing a direct link to a particular website with credible health information. Precisely identifying limitations in core operational behaviors central to eHealth literacy will help to inform more tailored and efficient eHealth literacy interventions that consider an individual’s perceptions of technology adoption and acceptability.

Compared with Millennials, older generations reported lower knowledge and self-efficacy in each of the factors captured by eHEALS. Specifically, adults belonging to Generation X and the Baby Boomers/Silent Generation had less confidence in their (1) awareness of online health information, (2) skills to locate online health information, and (3) ability to evaluate and act on health information once it is located online. This finding is consistent with previous literature stating that older adults have lower proficiency in eHealth literacy than their younger counterparts [18,22,25,26]. However, it is currently difficult to measure the degree to which specific eHealth literacy skills are deficient across different age segments. Our study helps to shed light on how to interpret eHEALS scores, such that information is gathered regarding which particular eHealth literacy skills are limited and the degree to which they are limited across age groups. eHEALS has strong potential to be used as the standard assessment tool for coordinating eHealth literacy training interventions based on these three discrete factors. For example, structured interventions could be delivered in three modules where skill-building activities aim to improve eHealth awareness, as well as information seeking and evaluation. Although older and younger adults respond differently to eHealth literacy interventions [85], these 3 factors (ie, skill sets) are central components of eHealth literacy, and thus should be considered in the planning, implementation, and evaluation of training interventions designed to improve the eHealth literacy of older adults through narrowing the chasm that currently exists between eHealth adoption and sustained use.

Finally, the results of this study provide implications for refining and updating the eHEALS. The brevity of eHEALS makes it an ideal scale for use in research and clinical care. However, it is necessary to ensure that there is an adequate number of items that correspond to each factor. Some measurement guidelines support the reliability of highly correlated factors that only comprise 2 items each [86,87]. However, other measurement standards recommend including at least 3 items per factor [88]. In the current study, the 3 eHEALS factors were correlated to a statistically significant degree. The strong factorial relationship allowed the model to function adequately with fewer items on Factors 1 and 2. This finding is contrary to the findings reported by Sudbury-Riley and colleagues [49], who found that only 1 latent factor (ie, online health information awareness) was best reflected by 2 eHEALS items, whereas factors related to information seeking and application (eg, knowing how to find and use online health information, self-efficacy to evaluate, and use online health information) were comprised of 3 items each. Further research is needed to develop unbiased items that sufficiently capture the theoretical underpinnings of eHealth literacy and its multidimensional constructs. Moreover, to account for the dynamic and interactive nature of eHealth [2], future research can build upon our findings to create and test new items that account for a fourth latent factor that captures “social” skills related to eHealth literacy.

Valid age group comparisons can be made with the 3-factor structure of eHEALS among Millennials, Generation Xers, and Baby Boomer/Silent Generation members. Results of this study add to the library of literature showing that older adults have significantly lower eHealth literacy scores as compared to younger adults. Specifically, this study supports that members of younger generations have a greater awareness of eHealth resources and more confidence in their information seeking and engagement skills on the Internet, as compared to older generations. The brevity of eHEALS coupled with its multi-dimensional structure can assist health care practitioners and researchers in tailoring eHealth literacy interventions designed to augment user performance on these relevant constructs. Furthermore, findings of this study have significant implications for more precisely measuring and improving eHealth literacy skills across the lifespan.