This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
The eHealth Literacy Scale (eHEALS) is a tool for the self-assessment of perceived comfort and skills in using the internet as a source for health-related information. Although evidence exists of the reliability and construct and structural validity of the scale, there is a lack of evidence in relation to what is proposed by Norman and Skinner in their theoretical lily model of eHealth literacy; in particular it is not clear whether having a higher level of health literacy can positively influence electronic health (eHealth) literacy as measured by the eHEALS.
Our study aim was to assess whether real-life experiences from studying or working in the health field, as a proxy of higher functional health literacy, correlate with self-referred eHealth literacy as measured by the eHEALS.
A Web-based survey was conducted among adults living in Northeast Italy using an Italian version of the eHEALS (IT-eHEALS). In order to be able to measure the effect of higher functional health literacy on eHealth literacy, we divided our sample into two groups, respectively characterized by studying or working experience in the health sector and by lack thereof. Mean differences between eHEALS were calculated using t test and effect size evaluated using Cohen d. To ensure the validity of the IT-eHEALS, we evaluated its psychometric properties (internal consistency and dimensionality) and construct validity (by evaluating its correlation with respondents age, gender, educational attainment, self-rated health, use of internet for health-related purposes, and working status).
A total of 868 respondents that completed the IT-eHEALS were included for analysis, of which 259 had working or studying experience in the health field. Mean (SD) eHEALS total score was 28.2 (6.2) for the whole sample, with statistically significant differences (
Our study demonstrates a sizeable effect of higher levels of functional health literacy on the eHEALS score, corroborating what was initially proposed by Norman and Skinner in the lily model of eHealth literacy.
Use of the internet for health-related purposes poses a particularly important challenge, as it has been shown that wrong or incomplete information available on the internet may have negative consequences for the user—including on doctor-patient relationships, participation in prevention and screening programs, or adherence to medical treatment [
In 2006, after three years of experimentation in a teenage health promotion program, Norman and Skinner developed the concept of eHealth literacy, drawing from the increasingly popular concept of health literacy. In the same year, the authors proposed both a theoretical model [
To further improve this first attempt to measure eHealth literacy, recent years have seen the development and validation of more comprehensive, and thus complex, eHealth literacy evaluation tools [
The sheer amount of unchecked health-related information on the internet can be seen either as a limit or as a resource by different respondents with different skills and experiences in the health field. Considering the subjective, self-referred nature of the eHEALS, one possibility could be that people with less knowledge in the health field would trust the information more as they would be less able to discern the real quality of their internet search findings, scoring higher in the scale. The aim of our study is to test the lily model, by assessing whether and to what extent differences in health literacy levels account for variations in the eHEALS score. To our knowledge, no prior study using the eHEALS explored whether the scale behaves as intended in populations with different sets of core skills or literacies as described in the lily model, in our case based on differences in health literacy levels. In their description of the lily model of eHealth literacy, Norman and Skinner use the definition of health literacy given by the American Medical Association [
In order to test our hypothesis, during November and December 2016 a Web-based survey was conducted by contacting people using two different recruitment strategies. Recruitment was performed using: (a) the mailing list of the student body (undergraduate and post-graduate) from the University of Udine (obtained with permission from the University), and (b) Facebook contacts of the public health research team members, who were then asked via Facebook to further disseminate the survey to their contacts. Decision to participate in the survey was voluntary and no incentives were offered to respondents. The survey was first pretested for usability and functionality by the members of the research team. The survey was administered using the software SurveyMonkey. All participants were asked to read and approve an informed consent form telling them that the study was managed by the University of Udine and that the survey would require approximately 15 minutes. As the survey did not collect any data that could be directly linked to participants’ sensitive data or information that could potentially affect their health, no approval by the Ethical Committee was deemed required under Italian legislation.
Collected measures covered socio-demographic characteristics (gender, age, highest educational level attained, working status), self-perceived health status, internet health-related behaviors (use for health-related search and frequency), working or studying experiences in the health sector, and an Italian adaptation of the eHEALS scale. Age was collected as a discrete variable, in number of years. Educational attainment was first collected using an 8-item scale, later aggregated into a 3-item scale in line with the aggregation methodology used by Eurostat in relation to International Standard Classification of Education levels [
To test our hypothesis, we selected the subsample of respondents who completed all of the 8 IT-eHEALS items. All collected data were screened to search for missing values or for any incorrect data inclusion. When not plausible, records were excluded from the analysis upon discussion among the research team. Then, the sample was divided into two groups based on having experiences of studying or working in the health sector or not. In this paper, we will refer to the group currently studying or working in the health sector as EHS+, and to the other as EHS-. Descriptive statistics (frequency, percentage, mean [SD]) were calculated for socio-demographic variables (gender, age, educational attainment, and working status), self-rated health, and internet health-related behaviors for all groups. A comparative analysis using Wilcoxon-Mann Whitney test and Chi-square (or Fisher Exact) Test, respectively for continuous and categorical variables, was conducted to detect statistically significant group differences (
Since we used a newly developed and adapted Italian version of the eHEALS (IT-eHEALS), we also assessed the scale by examining its psychometric properties and construct validity. Psychometric properties were examined by measuring internal consistency (Cronbach alpha) and conducting a principal component analysis to assess the dimensionality of the scale. Construct validity was assessed using a hypothesis testing approach. Based on prior studies, it was hypothesized that participants who (a) are younger [
Finally, differences between eHEALS means and SDs in the EHS+ and EHS- groups were calculated using
In total, the two internet surveys led to the recruitment of 1136, of which 868 completed all eight IT-eHEALS items, leading to a final sample of 868 respondents that were included for analysis.
IT-eHEALS showed a high degree of internal consistency with a Cronbach alpha of .90, with slight, negligible differences between the two groups (.87 in EHS-, .91 in EHS+). Principal Component Analysis in the whole sample confirmed the unidimensionality of the scale (eigenvalue=4.9 with 61.1% of variance explained). All IT-eHEALS items show high loadings on the first component (ranging from 0.68 to 0.83).
We did not find any difference in relation to gender. When assessing the whole sample, there was a significant difference for working status (
Descriptive and comparative analysis of study sample.
Variable | Whole sample (N=868), n (%) | EHS+a (N=259), n (%) | EHS-b (N=609), n (%) | ||
. 85 | |||||
Male | 231 (26.6) | 70 (27.0) | 161 (26.4) | ||
Female | 637 (73.4) | 189 (73.0) | 448 (73.6) | ||
.057 | |||||
Low | 22 (2.5) | 5 (1.9) | 17 (2.8) | ||
Middle | 457 (52.7) | 121 (46.7) | 336 (55.2) | ||
High | 383 (44.1) | 129 (49.8) | 254 (41.7) | ||
No response | 6 (0.7) | 4 (1.6) | 2 (0.3) | ||
<.001 | |||||
Working | 391 (45.1) | 139 (53.7) | 252 (41.4) | ||
Studying | 416 (47.1) | 113 (43.6) | 303 (49.7) | ||
Other | 61 (7.0) | 7 (2.7) | 54 (8.9) | ||
.27 | |||||
Very bad | 6 (0.7) | 1 (0.4) | 5 (0.8) | ||
Poor | 62 (7.1) | 21 (8.1) | 41 (6.7) | ||
Good | 455 (52.4) | 123 (47.5) | 332 (54.5) | ||
Very good | 281 (32.4) | 90 (34.7) | 191 (31.4) | ||
Excellent | 64 (7.4) | 24 (9.3) | 40 (6.6) | ||
<.001 | |||||
No more than 5-6 times/year | 282 (32.5) | 62 (23.9) | 220 (36.1) | ||
No more than 2-3 times/year | 135 (15.5) | 31 (12.0) | 104 (17.1) | ||
Once a month | 238 (27.4) | 58 (22.4) | 180 (29.6) | ||
Once a week | 109 (12.6) | 37 (14.3) | 72 (11.8) | ||
Several times a week | 104 (12.0) | 71 (27.4) | 33 (5.4) |
aEHS+: Group with studying or working experiences in the health sector.
bEHS-: Group without studying or working experiences in the health sector.
c
Spearman correlations between eHealth Literacy Scale total score for selected variables.
Variable | Whole sample | EHS+a | EHS-b | |||
Spearman correlation coefficient | Spearman correlation coefficient | Spearman correlation coefficient | ||||
Age | 0.11 | .002 | 0.22 | .001 | 0.02 | .65 |
Educational attainment | 0.11 | .001 | 0.19 | .002 | 0.06 | .13 |
Self-rated health | 0.07 | .038 | 0.14 | .024 | 0.02 | .70 |
Frequency of internet use for health | 0.28 | <.001 | 0.32 | <.001 | 0.15 | <.001 |
aEHS+: Group with studying or working experiences in the health sector.
bEHS-: Group without studying or working experiences in the health sector.
Italian version of eHealth Literacy Scale (eHEALS) items and total score statistics.
eHEALS score | Whole sample (N=868), mean (SD) | EHS+a (N=259), mean (SD) | EHS-b (N=609), mean (SD) | |
Item 1 | 3.8 (0.9) | 4.2 (0.8) | 3.6 (0.8) | <.001 |
Item 2 | 3.5 (0.9) | 3.9 (1.0) | 3.4 (0.9) | <.001 |
Item 3 | 3.6 (1.0) | 4.0 (0.9) | 3.4 (0.9) | <.001 |
Item 4 | 3.7 (0.9) | 4.1 (0.9) | 3.5 (0.9) | <.001 |
Item 5 | 3.7 (0.9) | 4.1 (0.9) | 3.6 (0.9) | <.001 |
Item 6 | 3.5 (1.2) | 4.2 (1.0) | 3.2 (1.1) | <.001 |
Item 7 | 3.8 (1.0) | 4.2 (0.8) | 3.6 (1.0) | <.001 |
Item 8 | 2.7 (1.2) | 3.2 (1.2) | 2.4 (1.1) | <.001 |
aEHS+: Group with studying or working experiences in the health sector.
bEHS-: Group without studying or working experiences in the health sector.
c
In our study we were able to demonstrate that real-life working or studying experiences in the health sector, as a proxy of higher levels of health literacy, positively correlate with self-referred eHealth literacy as measured by the eHEALS. This finding is in line with the original lily model of eHealth literacy proposed by Norman and Skinner, where eHealth literacy is described as the interconnection of different core skills, including health literacy. Our findings emphasize that there are different factors other than internet and computer skills that can lead to different results when measuring eHealth literacy.
Regarding the validity of the IT-eHEALS in the Italian population, we found high internal consistency, as shown by the Cronbach alpha and the inter-item correlation analysis, with comparable results with other translation of the eHEALS [
Our study has some limitations that should be acknowledged.
A first limitation of our study lies in the recruitment strategy used, which led to a study sample which is composed by respondents who are mostly young and highly educated, and therefore could not be considered representative of the adult Italian population, limiting the generalizability of our findings. While the English version of the scale has been applied in a variety of samples, most of the validating studies of the eHEALS in other languages have only been conducted among specific populations. Regarding gender, our sample has an overrepresentation of female respondents, so that our results shall be taken cautiously when trying to generalize to the general adult population. Also, it should be noted that the use of Facebook in our recruitment strategy made it impossible to assess number and characteristic of nonrespondents, an important limitation that should also be considered when interpreting results. While these are common shortcoming of similar validation studies, we believe that its composition characteristics (higher education level, younger age) are somewhat representative of the most active population of health information seekers in the internet, as reported by the latest 2017 EU Digital Scoreboard statistics for Italy about health information seeking in the general population (see
Another limitation of our study lies in the fact that we only included one measure of internet health-related behavior, as comparing different measures was outside the original scope of the study. While it should be acknowledged that this measure has not been previously validated, our results suggest that the two groups may indeed be different in terms of internet health-related behaviors, yet these should be further explored with a larger number of measures before reaching definitive conclusions on the health literacy role in explaining behavioral differences in this field. Also, we did not include any validated measures of either subjective or objective health literacy, which could have been used to quantitatively assess different levels of health literacy. Instead, we asked for real-life experiences in the health field as a proxy, which have been showed to correlate only with objective health literacy tests [
It must be noted that after our study was conducted, a validation study of another Italian version of the eHEALS (I-eHEALS) was published by Diviani et al, using a sample population of Italian-speaking Swiss respondents [
This study demonstrates that, as proposed in the lily model of eHealth literacy, eHEALS scale results are affected by a higher level of health literacy, measured via real-life experiences in the field of health as a proxy. We believe that this is an original result, which could be relevant in the current stage of scientific discussion regarding the use of the eHEALS and further advancements in measuring eHealth literacy. Despite its several limitations, and in absence of simple, easy-to-administer measurement tools, the eHEALS can still be considered a valid tool to assess self-perceived comfort and skills in using the internet for health-related purposes. It should still be used for comparison in the elaboration of new eHealth literacy measures, which should be designed including new items and different subscales in order to be able to capture all the proposed “literacies” of the construct [
eHealth Literacy Scale Italian versions.
Internet health-information seeking behavior in the Italian adult population.
analysis of variance
eHealth Literacy Scale
electornic health
Group with studying or working experiences in the health sector
Group without studying or working experiences in the health sector
Swiss-Italian version of the eHealth Literacy scale
Italian version of eHealth Literacy Scale
PDG, MP, ADO, LB, LA, and AC discussed and drafted the questionnaire for the surveys and discussed and approved the IT-eHEALS translation of the original eHEALS. AC and ADO managed data collection activities. PDG and GB analyzed collected data and interpreted results. PDG and GB drafted the manuscript, which was revised and approved by MP and SB. The study did not receive any funding.
None declared.