This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Parents often use digital media to search for information related to their children’s health. As the quantity and quality of digital sources meant specifically for parents expand, parents’ digital health literacy is increasingly important to process the information they retrieve. One of the earliest developed and widely used instruments to assess digital health literacy is the self-reported eHealth Literacy Scale (eHEALS). However, the eHEALS has not been psychometrically validated in a sample of parents. Given the inconsistency of the eHEALS underlying factor structure across previous reports, it is particularly important for validation to occur.
This study aimed to determine the factor structure of the German eHEALS measure in a sample of parents by adopting classic and modern psychometric approaches. In particular, this study sought to identify the eHEALS validity as a unidimensional index as well as the viability for potential subscales.
A cross-sectional design was used across two purposive sampling frames: online and paper administrations. Responses were collected between January 2018 and May 2018 from 703 Swiss-German parents. In addition to determining the sampling characteristics, we conducted exploratory factor analysis of the eHEALS by considering its ordinal structure using polychoric correlations. This analysis was performed separately for online–based and paper–based responses to examine the general factor strength of the eHEALS as a unidimensional index. Furthermore, item response theory (IRT) analyses were conducted by fitting eHEALS to a bifactor model to further inspect its unidimensionality and subscale viability.
Parents in both samples were predominantly mothers (622/703, 88.5%), highly educated (538/703, 76.9%), of Swiss nationality (489/703, 71.8%), and living with a partner (692/703, 98.4%). Factor analyses of the eHEALS indicated the presence of a strong general factor across both paper and online samples, and the Wilcoxon rank-sum test indicated that the eHEALS total sum score was not significantly different between the paper and online samples (
The German eHEALS evidenced good psychometric properties in a parent-specific study sample. Factor analyses indicated a strong general factor across purposively distinct sample frames (online and paper). IRT analyses validated the eHEALS as a unidimensional index while failing to find support for subscale usage.
Parents increasingly use digital sources when seeking information on their child's health [
Research on the eHealth literacy of parents is lacking. A study by Knapp et al [
The eHEALS was originally constructed for broad usage, as creators Norman and Skinner [
The study population consisted of a population-based sample of parents with children aged 1-24 months. The birth registries of Zürich and 5 municipalities in the canton of Zürich, which were selected using convenient sampling, provided randomly selected names and addresses of 2573 mothers who gave birth in the previous 24 months. Urban and rural municipalities were included to represent the urban/rural distribution in the German part of Switzerland (75%/25%). The ethical commission of the Canton of Zurich, based on the Swiss Federal Act on Research involving Human Beings, exempted the study from ethics review (BASEC Req-2017-00817).
The data were collected between January 2018 and May 2018. To increase the response rate, we applied a mixed-mode approach using online and paper versions of the questionnaire. The questionnaire consisted of three main parts: (1) sociodemographic characteristics of the parent and child, (2) digital media use in relation to the child’s health, and (3) health-related variables and eHealth literacy.
Parents received a postal invitation letter with a link to the online questionnaire. After the first postal reminder, parents received a paper questionnaire with the second and last reminder letters.
The eHEALS consists of 8 items (see
Example of past studies exploring the latent structure of the eHealth Literacy Scale (eHEALS) measure.
Variables | Studies | ||||
|
Norman & Skinner (2006) [ |
Soellner et al (2014) [ |
|||
|
|||||
|
Population | Canadian students | German students | ||
|
Sample size | 664 | 327 | ||
|
Age (years), range | 13-21 | 16-21 | ||
|
|
|
|||
|
Construction | Original | German translation, as reported here | ||
|
Structure | 1-factor | 2-factor | ||
|
|
||||
|
eHEALS item 1 | I know how to find helpful health resources on the Internet | Ich weiss, wie ich im Internet nützliche Gesundheitsinformationen findea | ||
|
eHEALS item 2 | I know how to use the Internet to answer my questions about health | Ich weiss, wie ich das Internet nutzen kann, um Antworten auf meine Fragen rund um das Thema Gesundheit zu bekommena | ||
|
eHEALS item 3 | I know what health resources are available on the Internet | Ich weiss, welche Quellen für Gesundheitsinformationen im Internet verfügbar sinda | ||
|
eHEALS item 4 | I know where to find helpful health resources on the Internet | Ich weiss, wo im Internet ich nützliche Gesundheitsinformationen finden kanna | ||
|
eHEALS item 5 | I know how to use the health information I find on the Internet to help me |
Ich weiss, wie ich Informationen aus dem Internet so nutzen kann, dass sie mir weiterhelfena | ||
|
eHEALS item 6 | I have the skills I need to evaluate the health resources I find on the Internet | Ich bin in der Lage, Informationen, die ich im Internet finde, kritisch zu bewertenb | ||
|
eHEALS item 7 | I can tell high quality health resources from low quality health resources on the Internet | Ich kann im Internet zuverlässige von Fragwürdigen Informationen unterscheidenb | ||
|
eHEALS item 8 | I feel confident in using information from the Internet to make health decisions. | Wenn ich gesundheitsbezogene Entscheidungen auf Basis von Informationen aus dem Internet treffe, fühle ich mich dabei sichera |
aInformation seeking.
bInformation appraisal.
We performed three different analyses to answer three distinct questions. The first analyses were descriptive and concerned differences in the sample characteristics. The second analyses were based on classical test theory and concerned the general factor strength for each sampling frame (online vs paper). The third analyses involved modern IRT and concerned unidimensionality assumptions and item-level bias across the online and paper administration samples.
Frequencies of the sociodemographic characteristics of the responding parents and their children were analyzed. Separately for the paper, online, and total samples, the single item and total sum eHEALS scores are reported as the median, skew, and mean. The total sum scores from the online and paper questionnaires were compared using the non-parametric Wilcoxon rank-sum test [
For Likert scales, it is recommended to consider their ordinal structure for factor analysis [
The detailed results of the EFA conducted with the psych package [
Previous findings have indicated unstable latent structures (number of eHEALS factors). In such a situation, the bifactor model helps to determine how useful it is to form subscales and examine if unidimensional IRT models can be fit to such multidimensional data [
Graphical representations of the eHealth unidimensional, correlated two-dimensional, and bi-factor two-dimensional models. The solid lines in the bi-factor model indicate unidimensional primacy over the residualized sub-dimensions (hashed arrows). eHealth: electronic health; eHeal: eHealth Literacy.
A total of 842 parents or caretakers responded to the survey, and we excluded 73 responses during the data cleaning process for the following reasons: incomplete questionnaire (n=31), missing answers to key questions on parental digital health information seeking (not including the eHEALS items; n=40), non-plausibility of key questions (n=1), and duplicate entry (n=1). This resulted in 769 observations corresponding to a response rate of 30% for the overall study. The online questionnaire was completed by 429 participants (429/769, 56%), and 340 participants (340/769, 44%) completed the paper questionnaire. For the analysis of the eHEALS, 67 additional observations had to be discarded because 52 had missing values for all eHEALS items and 15 had missing values for single eHEALS items.
This led to a final online sample of 388 participants and a final paper sample of 315 participants.
Summary of the sample characteristics, N=703.
Characteristic | Participants, n (%) | |||
|
|
|||
|
Mother | 622 (88.5) | ||
Father | 78 (11.1) | |||
|
Other | 3 (0.4) | ||
Age (years) | 35.7 (4.3)a | |||
|
|
|||
|
Lower education | 162 (23.1) | ||
Higher education | 538 (76.9) | |||
|
|
|||
|
Swiss | 489 (71.8) | ||
Other | 192 (28.2) | |||
|
|
|||
Yes | ||||
No | 11 (1.6) | |||
|
|
|||
|
<4500 | 27 (4.2) | ||
4500-6000 | 94 (14.5) | |||
6000-9000 | 233 (36.0) | |||
>9000 | 294 (45.4) | |||
|
|
|||
|
Female | 349 (49.9) | ||
Male | 350 (50.1) | |||
Child’s age (months) | 14.8 (7.1)a | |||
|
|
|||
|
Yes | 353 (51.2) | ||
No | 337 (48.8) | |||
Digital media use scoreb | 7.88 (4.13)a | |||
eHEALS total sum score | 29.0 (5.9)a |
aMean (SD).
bSum score on how often parents use several digital media for general child health and development (ranging from 0-24).
Concerning the eHEALS items, there were no differences in the individual item responses between the paper and online modes, except for item 3, where the online sample yielded a lower median. The distributions for all items for both the paper and online samples were slightly negatively skewed. As there was no significant difference in the eHEALS total sum scores between the online and paper samples (
Descriptive statistics of the individual eHealth Literacy Scale (eHEALS) items and total sum score for the online, paper, and total samples.
eHEALS item | Online (n=388) | Paper (n=315) | Total sample (n=703) | ||||||
|
Median | Skew | Median | Skew | Median | Mean (SD) | |||
Item 1 | 4 | –1.02 | 4 | –0.76 | 3.7 (1.1) | ||||
Item 2 | 4 | –0.91 | 4 | –0.95 | 4 | 3.7 (1.0) | N/A | ||
Item 3 | 3 | –0.55 | 4 | –0.55 | 4 | 3.4 (1.0) | N/A | ||
Item 4 | 4 | –0.57 | 4 | –0.61 | 4 | 3.5 (1.0) | N/A | ||
Item 5 | 4 | –0.87 | 4 | –0.95 | 4 | 3.7 (0.9) | N/A | ||
Item 6 | 4 | –1.40 | 4 | –1.23 | 4 | 4.2 (0.9) | N/A | ||
Item 7 | 4 | –0.77 | 4 | –0.81 | 4 | 3.9 (0.9) | N/A | ||
Item 8 | 3 | –0.24 | 3 | –0.21 | 3 | 3.0 (1.1) | N/A | ||
eHEALS total sum score | 30 | –0.67 | 29 | –0.76 | 30 | 28.5 (6.2) | 0.12a (0.12b) |
aMedian test.
bWilcoxon rank-sum test.
Given the non-significant difference in the eHEALS total sum scores between theoretically distinct sampling frames (paper and online collection methods), a series of exploratory factor analyses were conducted to examine the strength of the general factor across the samples. As shown in
When considered collectively with the non-significant difference in eHEALS total scores between sampling frames, we used IRT to test the unidimensionality assumptions with the total eHEALS sample.
Exploratory factor scree plots to examine the strength of the general factors across the online, paper, and total samples.
Separate bifactor models with two group factors and three group factors were computed. A direct model comparison between the bifactors indicated the two-group factor model exhibited significantly greater fit than the three-group factor model (χ21=58.4,
Furthermore, the item loadings between the unidimensional and bifactor models were compared to determine the impact on the bias from ignoring suspected multidimensionality (literally, comparing across models with and without additional dimensions).
The average relative parameter bias (0.09;
Item response theory electronic health item loadings, N=703.
Item | Unidimensional model | Bifactor model with two group factors | ||
|
|
General Factor | Factor 1 | Factor 2 |
1 | 0.81 | 0.87 | –0.09 | N/A |
2 | 0.88 | 0.94 | –0.07 | N/A |
3 | 0.89 | 0.81 | 0.40 | N/A |
4 | 0.91 | 0.82 | 0.56 | N/A |
5 | 0.88 | 0.85 | 0.11 | N/A |
6 | 0.57 | 0.49 | N/A | 0.74 |
7 | 0.68 | 0.6 | N/A | 0.68 |
8 | 0.66 | 0.63 | 0.10 | N/A |
Item response theory electronic health unidimensionality indices, N=703.
Unidimensionality index | Value |
ECVa | 0.76 |
Omega reliability | 0.99 |
Hierarchical omega | 0.92 |
H replicability | 0.95 |
Factor determinacy | 0.99 |
ARPBb | 0.09 |
IECVc (number of items >0.80) | 5 |
aECV: estimated common variance.
bARPB: average relative parameter bias.
cIECV: item estimated common variance.
To verify this inference, first, we examined the correlation between the eHEALS and a meaningful substantive variable from the survey with the parents (sum score on how often parents use several digital media for general child health and development), which was significant in the expected direction (r=0.29,
The findings of this study support the usefulness of the eHEALS measure as a unidimensional index for further studies. Specifically, we found a strong general factor of the eHEALS across distinct sampling frames as well as adequate reliability. Furthermore, the IRT analyses indicated minimal distortion of the primary factor from ignoring potential multidimensionality, and subscale reliabilities were inadequate to recommend further usage.
With respect to the EFA, this study used a different methodology to add to the current discussion of the eHEALS factor structure. Norman and Skinner [
Given the construction and theorized application of eHEALS as a broad construct, our IRT analyses included a bifactor model. Consistent with previous IRT analyses, our findings indicated unidimensionality of the eHEALS [
In summary, the results support the broad but unidimensional factor structure of the German eHEALS. Our ordinal factor analysis supports the presence of a strong general factor. Furthermore, the item response theory analysis using bifactor models with one general factor and two or three group factors showed that the model with two group factors fitted better than the one with three factors. Comparing this bifactor model with two group factors with the unidimensional loadings did not suggest a substantial difference in primary loadings. Finally, we found no support for using eHEALS subscales.
The use of subscales in previous research [
This study has some limitations. Although the parents were asked about their own eHealth literacy, it is likely that the questions on child health prompted the parents to answer the eHEALS items from the perspective of child health. This would explain the parents’ reluctance to make decisions based on internet-based health information (item 8 of the eHEALS). Therefore, in comparison with studies on adult eHealth literacy, parental eHealth literacy might be lower. However, the high information needs of parents, especially right after birth, might have increased eHealth literacy simply through practice and experience. Regarding the sample characteristics, the generalizability of our findings might be limited by the fairly low response rate (769/842, 30%) and the uniquely high socioeconomic status.
Another limitation is that measurement invariance was not assessed in terms of the participants’ individual characteristics. For example, future studies could focus specifically on the generalizability by gender of our proposed unidimensional eHEALS. This was not the focus of the current study and will be addressed in further analyses. It is important, furthermore, for future researchers to consider the relevance of their samples when studying eHEALS measurement properties. This study aimed to extend the application of the eHEALS among new parents.
This study suggests that the German eHEALS possesses a broad, unidimensional factor structure among Swiss-German parents. Although the two samples differed with respect to participant characteristics such as age, education, and income, we failed to find a significant difference in the eHEALS total sum scores. The underrepresentation of participants of lower socioeconomic status, not only in our study but also in many other studies on digital health, warrants future studies to over-sample this population. We found similar factor structures and item properties irrespective of application mode. That is, the EFAs suggested a strong general factor. Finally, bifactor modeling did not outperform the unidimensional model, and subscales were unsupported because of low reliability. While using the total sum score is appropriate to assess eHealth literacy, further development and refinement of the eHEALS are proposed to address specific sub-domains of eHealth literacy. For any sample, practitioners should use only the eHEALS total score, and future research aiming to utilize subscales should expand the eHEALS item pool for empirical testing.
Detailed protocol of the exploratory factor analysis with polychoric correlation matrix.
Potential core set of the German eHEALS and differential item functioning analysis (DIF).
exploratory factor analysis.
eHealth Literacy Scale.
item response theory.
The authors thank the municipalities who provided the parents’ addresses, all participating parents for their valuable engagement, and our colleague Dominik Robin for his support in study design. This study was funded by the Käthe Zingg-Schwichtenberg Fund (KZS) of the Swiss Academy of Medical Sciences (SAMS). The authors of this manuscript are independent of the funding agencies. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
None declared.