Use of the internet for health-related purposes poses a particularly important challenge, as it has been shown that wrong or incomplete information available on the internet may have negative consequences for the user—including on doctor-patient relationships, participation in prevention and screening programs, or adherence to medical treatment [1]. Today, the availability and accessibility of quality health-related internet information is still an issue, and agreement on a specified set of quality standards for health websites has recently been proposed as a new public health priority [2]. The problem of providing quality health-related information has become even more complex in the current Web 2.0 environment, as the search strategy for relevant information depends not only on the searcher’s ability, but also on the influence of intermediators and apomediators, with the latter effectively pushing the search towards or away from relevant items [3].

In 2006, after three years of experimentation in a teenage health promotion program, Norman and Skinner developed the concept of eHealth literacy, drawing from the increasingly popular concept of health literacy. In the same year, the authors proposed both a theoretical model [4] and a tool to measure the new construct [5]. In their view, eHealth literacy was defined as “the ability to seek, find, understand, and appraise health information from electronic sources and apply the knowledge gained to addressing or solving a health problem.” The proposed model, called “lily model”, described eHealth literacy as the interplay of six core skills or literacies (traditional literacy, health literacy, information literacy, scientific literacy, media literacy, and computer literacy). The measurement tool, called eHealth literacy scale (eHEALS), was based on the principles of the social cognitive theory and self-efficacy theory, using Likert scales for self-assessed, subjective responses, so that measures should be considered as precursors of behavior change and skill development [6]. In this sense the eHEALS can be considered a measure of subjective, self-assessed eHealth literacy.

To further improve this first attempt to measure eHealth literacy, recent years have seen the development and validation of more comprehensive, and thus complex, eHealth literacy evaluation tools [7-9]. It should be noted, though, that despite all the possible issues coming from the simple eight-item structure of the eHEALS, its simplicity is also a strength of the tool. In fact, at the time of its development, the eHEALS was explicitly conceived so that it would have been easy to administer, taking into consideration the expressed needs of health professionals who said that they wouldn’t use a long instrument in their practice [10]. This “strength in simplicity” facilitated the adoption of the eHEALS, and its widespread use has been highlighted in the findings of several literature reviews. In 2012 Collins and colleagues reviewed the use of health literacy screening tools in eHealth applications [11] and found that the eHEALS was the most used for the purpose of developing a computer-based instrument to screen individuals accessing eHealth applications, alone or in combination with other screening tools for health literacy. In a 2015 review of existing tools to measure eHealth literacy and their use by Karnoe and Kayser [12], the authors found that, out of eight different tools for measuring eHealth literacy, only the eHEALS had been used in studies other than the one it was originally published in. The same authors argue that the eHEALS, while easy to administer, provides a measure that is not able to pinpoint whether inadequate eHealth literacy is a result of insufficient health literacy, digital literacy, or a combination hereof. In other words, it is still not clear whether a higher level of self-referred eHealth literacy (using the eHEALS), is correctly due to differences in levels of functional health literacy or is just a result of high levels of perceived self-efficacy.

The sheer amount of unchecked health-related information on the internet can be seen either as a limit or as a resource by different respondents with different skills and experiences in the health field. Considering the subjective, self-referred nature of the eHEALS, one possibility could be that people with less knowledge in the health field would trust the information more as they would be less able to discern the real quality of their internet search findings, scoring higher in the scale. The aim of our study is to test the lily model, by assessing whether and to what extent differences in health literacy levels account for variations in the eHEALS score. To our knowledge, no prior study using the eHEALS explored whether the scale behaves as intended in populations with different sets of core skills or literacies as described in the lily model, in our case based on differences in health literacy levels. In their description of the lily model of eHealth literacy, Norman and Skinner use the definition of health literacy given by the American Medical Association [13], which can be arguably referred to as a basic “functional” level of health literacy [14]. There are several possible measures of functional health literacy using measurement tools that are grounded on different theories. Yet, functional measures of health literacy have been shown to correlate with studying or working experiences in the health field [15], with physicians, researcher in the health field, and nurses showing higher health literacy levels in comparison with the general population. Therefore, we chose to recruit a large sample of respondents, divided into two groups, by asking them whether they had real-life experiences in studying or working in the health sector (eg, physicians, nurses, health alliance professionals). By doing so, we were able to compare a highly health-literate group with the rest of the general population.

In order to test our hypothesis, during November and December 2016 a Web-based survey was conducted by contacting people using two different recruitment strategies. Recruitment was performed using: (a) the mailing list of the student body (undergraduate and post-graduate) from the University of Udine (obtained with permission from the University), and (b) Facebook contacts of the public health research team members, who were then asked via Facebook to further disseminate the survey to their contacts. Decision to participate in the survey was voluntary and no incentives were offered to respondents. The survey was first pretested for usability and functionality by the members of the research team. The survey was administered using the software SurveyMonkey. All participants were asked to read and approve an informed consent form telling them that the study was managed by the University of Udine and that the survey would require approximately 15 minutes. As the survey did not collect any data that could be directly linked to participants’ sensitive data or information that could potentially affect their health, no approval by the Ethical Committee was deemed required under Italian legislation.

Collected measures covered socio-demographic characteristics (gender, age, highest educational level attained, working status), self-perceived health status, internet health-related behaviors (use for health-related search and frequency), working or studying experiences in the health sector, and an Italian adaptation of the eHEALS scale. Age was collected as a discrete variable, in number of years. Educational attainment was first collected using an 8-item scale, later aggregated into a 3-item scale in line with the aggregation methodology used by Eurostat in relation to International Standard Classification of Education levels [16]. The final set of education levels used for analysis were: (low) 8th grade or lower, (middle) 9-13th grade, (high) university degree or higher. Working status was collected asking participants whether they were currently working, studying, or neither working nor studying (classified as “other”). Self-rated health was collected using a 5-item Likert scale, ranging from “very bad” to “excellent,” with the midpoint rated as “good.” Health-related internet use was measured asking the frequency of internet use for health-related purposes (using a 5-item Likert scale ranging from “not more than 5-6 times a year” to “several times a week”). To differentiate for real-life experiences in the health sector, participants were asked whether they had experiences in studying or working in the health sector using a yes/no question. Regarding the Italian version of the eHEALS, we were unable to retrieve a previously reported version of the tool (I-eHEALS) presented in a conference abstract by De Caro et al [17] (via request to the corresponding author), and a new Italian translation of the 8 eHEALS items was produced by the research team (IT-eHEALS). The translation process was carried out following established good practices [18]: the original English tool was initially distributed among the research team, producing a first set of translations that were later merged into a single draft version. The draft of the Italian instrument was then retranslated into English by an interpreter and reviewed by the research team for correctness. Translated items were pretested for comprehensibility on a small sample of Italian adults (N=24) and items were adjusted accordingly. Like the original version of the test, the IT-eHEALS is composed of 8 items measured with a 5-point Likert scale. For every respondent of the sample that completed all 8 IT-eHEALS items, the total score ranges from 8 to 40 (calculated by adding up the single items’ scores), with a higher score indicating a higher self-referred eHealth literacy.

In total, the two internet surveys led to the recruitment of 1136, of which 868 completed all eight IT-eHEALS items, leading to a final sample of 868 respondents that were included for analysis. Table 1 shows the socio-demographic characteristics of the whole sample and differences between the EHS+ and EHS- groups. The two groups differ significantly (P<.001) in relation to working status and frequency of internet use for health-related purposes. In the EHS+ group, most of the respondents are working (139/259, 53.7%), while in EHS- the majority are studying (303/609, 49.7%). Regarding internet use for health-related purposes, 27.4% (71/259) of EHS+ respondents use the internet more than once a week, while only 5.42% (33/609) of the EHS- respondents do so, suggesting differences in health-related internet behaviors between the two groups. Also, respondents in the EHS+ group are significantly older, with a mean age of 31.5±12.1 years vs 28.7±9.7 years for the EHS- group (P=.008).

IT-eHEALS showed a high degree of internal consistency with a Cronbach alpha of .90, with slight, negligible differences between the two groups (.87 in EHS-, .91 in EHS+). Principal Component Analysis in the whole sample confirmed the unidimensionality of the scale (eigenvalue=4.9 with 61.1% of variance explained). All IT-eHEALS items show high loadings on the first component (ranging from 0.68 to 0.83). Table 2 shows Spearman correlation coefficients with age and educational attainment. Correlation coefficients of total mean scores on the IT-eHEALS with selected variables are significant but low, with the exception of age, educational attainment, and self-rated health in EHS-. The correlation with frequency of internet use for health-related purposes was significant in both groups.

We did not find any difference in relation to gender. When assessing the whole sample, there was a significant difference for working status (P=.03) that was not present when considering EHS+ and EHS- separately in both groups.

Table 3 shows the mean (SD) item score and the statistical significance of the difference between the EHS+ and EHS- groups (see Multimedia Appendix 1 for item descriptions). Considering the whole sample of IT-eHEALS respondents, mean values for items range from 3.8 (item 1) to 2.7 (item 8). Differences between the two groups were significant for all IT-eHEALS items (P<.001), with the mean (SD) total score significantly higher in EHS+ compared to EHS- (31.9 [5.9] vs 26.7 [5.6], P<.001). The standardized mean difference (Cohen d) was 0.9, demonstrating a sizeable effect of higher levels of functional health literacy on the eHEALS score.

Our study has some limitations that should be acknowledged.

This study demonstrates that, as proposed in the lily model of eHealth literacy, eHEALS scale results are affected by a higher level of health literacy, measured via real-life experiences in the field of health as a proxy. We believe that this is an original result, which could be relevant in the current stage of scientific discussion regarding the use of the eHEALS and further advancements in measuring eHealth literacy. Despite its several limitations, and in absence of simple, easy-to-administer measurement tools, the eHEALS can still be considered a valid tool to assess self-perceived comfort and skills in using the internet for health-related purposes. It should still be used for comparison in the elaboration of new eHealth literacy measures, which should be designed including new items and different subscales in order to be able to capture all the proposed “literacies” of the construct [4]. For these reasons, we believe that the absence of correlation of the eHEALS with objectively measured internet related skills as found by different authors does not undermine the validity and the usability of the scale per se, and that the eHEALS can still be applied in clinical and health promotion activities, for example to identify different needs for the participants to an eHealth intervention or to evaluate intervention results.