Published on in Vol 22, No 3 (2020): March

Preprints (earlier versions) of this paper are available at, first published .
Electronic Health Literacy in Swiss-German Parents: Cross-Sectional Study of eHealth Literacy Scale Unidimensionality

Electronic Health Literacy in Swiss-German Parents: Cross-Sectional Study of eHealth Literacy Scale Unidimensionality

Electronic Health Literacy in Swiss-German Parents: Cross-Sectional Study of eHealth Literacy Scale Unidimensionality

Original Paper

1Institute of Health Sciences, Department of Health, Zurich University of Applied Sciences, Winterthur, Switzerland

2University of Basel, Basel, Switzerland

Corresponding Author:

Matthew J Kerry, BA, MS, PhD

Institute of Health Sciences

Department of Health

Zurich University of Applied Sciences

Technikumstrasse 71

Winterthur, 8400


Phone: 41 58 934 41 79


Background: Parents often use digital media to search for information related to their children’s health. As the quantity and quality of digital sources meant specifically for parents expand, parents’ digital health literacy is increasingly important to process the information they retrieve. One of the earliest developed and widely used instruments to assess digital health literacy is the self-reported eHealth Literacy Scale (eHEALS). However, the eHEALS has not been psychometrically validated in a sample of parents. Given the inconsistency of the eHEALS underlying factor structure across previous reports, it is particularly important for validation to occur.

Objective: This study aimed to determine the factor structure of the German eHEALS measure in a sample of parents by adopting classic and modern psychometric approaches. In particular, this study sought to identify the eHEALS validity as a unidimensional index as well as the viability for potential subscales.

Methods: A cross-sectional design was used across two purposive sampling frames: online and paper administrations. Responses were collected between January 2018 and May 2018 from 703 Swiss-German parents. In addition to determining the sampling characteristics, we conducted exploratory factor analysis of the eHEALS by considering its ordinal structure using polychoric correlations. This analysis was performed separately for online–based and paper–based responses to examine the general factor strength of the eHEALS as a unidimensional index. Furthermore, item response theory (IRT) analyses were conducted by fitting eHEALS to a bifactor model to further inspect its unidimensionality and subscale viability.

Results: Parents in both samples were predominantly mothers (622/703, 88.5%), highly educated (538/703, 76.9%), of Swiss nationality (489/703, 71.8%), and living with a partner (692/703, 98.4%). Factor analyses of the eHEALS indicated the presence of a strong general factor across both paper and online samples, and the Wilcoxon rank-sum test indicated that the eHEALS total sum score was not significantly different between the paper and online samples (P=.12). Finally, the IRT analyses indicated negligible multidimensionality, insufficient subscale reliability after accounting for the eHEALS general factor, and a reduced subset of items that could serve as a unidimensional index of the eHEALS across the paper and online samples.

Conclusions: The German eHEALS evidenced good psychometric properties in a parent-specific study sample. Factor analyses indicated a strong general factor across purposively distinct sample frames (online and paper). IRT analyses validated the eHEALS as a unidimensional index while failing to find support for subscale usage.

J Med Internet Res 2020;22(3):e14492



Parents increasingly use digital sources when seeking information on their child's health [1-3]. Through their accessibility, digital sources offer the opportunity for parents to feel empowered [4] (eg, to verify information received from health professionals), consider alternative treatment options, and develop communal networks with other families and patients with a common disease or condition. However, because the quality and reliability of information from digital sources vary substantially [5], the information can be overwhelming and cause insecurity or anxiety [6-8]. Therefore, eHealth literacy of parents is critical to maximize the potential benefits of digital media for children’s health. eHealth literacy has been defined as “a set of skills required to effectively engage information technology for health” [9].

Research on the eHealth literacy of parents is lacking. A study by Knapp et al [10] in Florida showed that low-income parents of children with special health needs had high levels of internet use for information purposes. However, half of the study participants had difficulties separating high-quality from low-quality information and were not confident using the internet. Similar findings were found in a study of parents whose children had life-threatening illnesses [11]. Both studies used the eHealth Literacy Scale (eHEALS) [9]. Both studies are also older, considering the fast-evolving context of internet acculturation. The eHEALS was developed by Norman and Skinner [12], who defined eHealth literacy as “the ability to seek, find, understand, and appraise health information from electronic sources and apply the knowledge gained to addressing or solving a health problem.” In this regard, the eHEALS pertains to the critical consumption of extant internet content rather than the creation of new content. The eHEALS is a widely applied instrument that has been validated internationally in diverse languages [13]. Adequate internal consistency has been found [13]. However, van der Vaart et al [14] reported weak correlations between the eHEALS scores and tasks on an eHealth performance test. In contrast, more recent research found significant associations between perceptions and performance [15]. In addition, inconsistent results for the factor structure of the eHEALS have been reported. Norman and Skinner [9] proposed a one-factor structure for the original English eHEALS. For the German eHEALS, Soellner et al [16] determined a two-factor structure using confirmatory factor analysis. For an Italian version of the eHEALS applied in the Italian-speaking area of Switzerland, the researchers [17] recommended using the eHEALS total sum score after applying Rasch (item response theory [IRT]) modeling. However, a study with patients at risk of cardiovascular disease in Australia also applied Rasch modeling and concluded that the eHEALS captures different aspects of eHealth literacy, which may have to be scored separately [18]. Neter et al [19] reported a different two-factor solution for adults in Israel aged at least 21 years than the two-factor solution used by Soellner et al [16]. Moreover, two recent studies developed a three-factor solution for the English eHEALS: awareness, skills, and evaluation [20,21]. A three-factor solution was also reported in an IRT analysis of eHEALS, although the authors noted that substantially high interfactor correlations “support an overarching structure of eHEALS” [22]. Notably, both studies implementing IRT analyses of the eHEALS found measurement properties (item difficulties) reflective of their study samples. That is, Diviani et al [17] found wide variability in item difficulties in a broad sample of people aged 16-71 years, and Stellefson et al [22] found high item difficulties in a sample narrowed to older adults.

The eHEALS was originally constructed for broad usage, as creators Norman and Skinner [9] state, “this article describes the development and psychometric evaluation of a measure of eHealth literacy designed for broad use in supporting consumer eHealth in public health and clinical care” (p2). Subsequent research has reported unstable latent factor structures underlying the eHEALS measure, despite the conventional understanding that “broader constructs are stabilized with broad factors” [23]. Given the inconsistent results for the eHEALS, we aimed to establish the psychometric structure of the eHEALS in parents participating in the Digital Parental Counselors study (in German: Digitale Elternratgeber), which investigated digital media use by parents for their children’s health in the German-speaking area of Switzerland [24]. Specifically, we aimed to explore the factor structure of the German eHEALS and to assess the viability of subscales using IRT and bifactor modeling. We also addressed the methodological issues concerning the handling of Likert scales and the use of factor analysis experienced with previous research of the eHEALS.

Study Population

The study population consisted of a population-based sample of parents with children aged 1-24 months. The birth registries of Zürich and 5 municipalities in the canton of Zürich, which were selected using convenient sampling, provided randomly selected names and addresses of 2573 mothers who gave birth in the previous 24 months. Urban and rural municipalities were included to represent the urban/rural distribution in the German part of Switzerland (75%/25%). The ethical commission of the Canton of Zurich, based on the Swiss Federal Act on Research involving Human Beings, exempted the study from ethics review (BASEC Req-2017-00817).

Data Collection

The data were collected between January 2018 and May 2018. To increase the response rate, we applied a mixed-mode approach using online and paper versions of the questionnaire. The questionnaire consisted of three main parts: (1) sociodemographic characteristics of the parent and child, (2) digital media use in relation to the child’s health, and (3) health-related variables and eHealth literacy.

Parents received a postal invitation letter with a link to the online questionnaire. After the first postal reminder, parents received a paper questionnaire with the second and last reminder letters.

eHealth Literacy Scale

The eHEALS consists of 8 items (see Table 1). Responses are provided using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree), with total scores ranging from 8 to 40 points. Higher scores reflect higher eHealth literacy. The eHEALS was developed based on the concept that eHealth literacy is composed of core skills grouped into analytical skills such as media literacy and context-specific skills such as health literacy [12]. The eHEALS does not measure these skills directly, but rather “the consumer's perceived skills and comfort with eHealth” [9]. For this study, we used the German eHEALS version developed by Soellner et al [16], who translated and cross-validated the original English version by Norman and Skinner [9] in a German sample.

Table 1. Example of past studies exploring the latent structure of the eHealth Literacy Scale (eHEALS) measure.

Norman & Skinner (2006) [9]Soellner et al (2014) [16]
Sample characteristics

PopulationCanadian studentsGerman students

Sample size664327

Age (years), range13-2116-21
Factor Solution

ConstructionOriginalGerman translation, as reported here


eHEALS item 1I know how to find helpful health resources on the InternetIch weiss, wie ich im Internet nützliche Gesundheitsinformationen findea

eHEALS item 2I know how to use the Internet to answer my questions about healthIch weiss, wie ich das Internet nutzen kann, um Antworten auf meine Fragen rund um das Thema Gesundheit zu bekommena

eHEALS item 3I know what health resources are available on the InternetIch weiss, welche Quellen für Gesundheitsinformationen im Internet verfügbar sinda

eHEALS item 4I know where to find helpful health resources on the InternetIch weiss, wo im Internet ich nützliche Gesundheitsinformationen finden kanna

eHEALS item 5I know how to use the health information I find on the Internet to help me
Ich weiss, wie ich Informationen aus dem Internet so nutzen kann, dass sie mir weiterhelfena

eHEALS item 6I have the skills I need to evaluate the health resources I find on the InternetIch bin in der Lage, Informationen, die ich im Internet finde, kritisch zu bewertenb

eHEALS item 7I can tell high quality health resources from low quality health resources on the InternetIch kann im Internet zuverlässige von Fragwürdigen Informationen unterscheidenb

eHEALS item 8I feel confident in using information from the Internet to make health decisions.Wenn ich gesundheitsbezogene Entscheidungen auf Basis von Informationen aus dem Internet treffe, fühle ich mich dabei sichera

aInformation seeking.

bInformation appraisal.

Data Analysis

We performed three different analyses to answer three distinct questions. The first analyses were descriptive and concerned differences in the sample characteristics. The second analyses were based on classical test theory and concerned the general factor strength for each sampling frame (online vs paper). The third analyses involved modern IRT and concerned unidimensionality assumptions and item-level bias across the online and paper administration samples.

Descriptive Analysis

Frequencies of the sociodemographic characteristics of the responding parents and their children were analyzed. Separately for the paper, online, and total samples, the single item and total sum eHEALS scores are reported as the median, skew, and mean. The total sum scores from the online and paper questionnaires were compared using the non-parametric Wilcoxon rank-sum test [25] prior to merging the data. All descriptive analyses were computed with Stata 15.0 (StataCorp LLC, College Station, TX).

Classical Test Theory Analysis

For Likert scales, it is recommended to consider their ordinal structure for factor analysis [26]. As the conventional exploratory factor analysis (EFA) treats variables in a metric manner, polychoric correlations were computed to take into account the ordinal structure of the eHEALS items (see Multimedia Appendix 1).

The detailed results of the EFA conducted with the psych package [27] in R (R Foundation for Statistical Computing, Vienna, Austria) are displayed in Multimedia Appendix 1. Furthermore, a series of parallel analyses were conducted to determine the single-factor strength of the eHEALS measure across the sampling frames [28] using SPSS version 25.0 (IBM Corp, Armonk, NY). Finally, the internal consistency of the eHEALS scale was assessed using the McDonald’s omega coefficient [29].

Item Response Theory Analysis

Previous findings have indicated unstable latent structures (number of eHEALS factors). In such a situation, the bifactor model helps to determine how useful it is to form subscales and examine if unidimensional IRT models can be fit to such multidimensional data [30]. We therefore inspected the eHEALS using a bifactor model. In a bifactor model, a general factor is generated through all test items. Additionally, group factors are established out of the residual variance shared by subsets of items [31] (see Figure 1). In addition to the bifactor model, further IRT understanding based on the internal psychometric structure of the eHEALS was specifically examined in terms of general factor strength (appropriate unidimensional scores) and substantive multidimensionality (item bias). IRTPRO v 4.2 (Scientific Software International, Skokie, IL) was used to run the IRT analyses, and the unidimensional indices were computed using the Bifactor Indices Calculator [32], including McDonald´s omega [29].

Figure 1. Graphical representations of the eHealth unidimensional, correlated two-dimensional, and bi-factor two-dimensional models. The solid lines in the bi-factor model indicate unidimensional primacy over the residualized sub-dimensions (hashed arrows). eHealth: electronic health; eHeal: eHealth Literacy.
View this figure

Descriptive Analysis

A total of 842 parents or caretakers responded to the survey, and we excluded 73 responses during the data cleaning process for the following reasons: incomplete questionnaire (n=31), missing answers to key questions on parental digital health information seeking (not including the eHEALS items; n=40), non-plausibility of key questions (n=1), and duplicate entry (n=1). This resulted in 769 observations corresponding to a response rate of 30% for the overall study. The online questionnaire was completed by 429 participants (429/769, 56%), and 340 participants (340/769, 44%) completed the paper questionnaire. For the analysis of the eHEALS, 67 additional observations had to be discarded because 52 had missing values for all eHEALS items and 15 had missing values for single eHEALS items.

This led to a final online sample of 388 participants and a final paper sample of 315 participants. Table 2 provides the summary descriptive statistics of the whole sample (N=703). Of the sample, 88.5% (622/703) of the participants were mothers, 76.9% (538/703) reported a university degree or higher vocational education as their highest educational level, and 45.4% (294/703) earned a monthly income >9000 Swiss Francs (US $9080; €8020). The majority (489/703, 71.8%) were Swiss, and almost all study participants (692/703, 98.4%) indicated they lived with a partner.

Table 2. Summary of the sample characteristics, N=703.
CharacteristicParticipants, n (%)

Parental sex

Mother622 (88.5)
Father78 (11.1)

Other3 (0.4)
Age (years)35.7 (4.3)a
Education level

Lower education162 (23.1)
Higher education538 (76.9)

Swiss489 (71.8)
Other192 (28.2)
Living with a partner
692 (98.4)
No11 (1.6)
Household net monthly income (CHF)

<450027 (4.2)
4500-600094 (14.5)
6000-9000233 (36.0)
>9000294 (45.4)
Child’s sex

Female349 (49.9)
Male350 (50.1)
Child’s age (months)14.8 (7.1)a
First child

Yes353 (51.2)
No337 (48.8)
Digital media use scoreb7.88 (4.13)a
eHEALS total sum score29.0 (5.9)a

aMean (SD).

bSum score on how often parents use several digital media for general child health and development (ranging from 0-24).

Concerning the eHEALS items, there were no differences in the individual item responses between the paper and online modes, except for item 3, where the online sample yielded a lower median. The distributions for all items for both the paper and online samples were slightly negatively skewed. As there was no significant difference in the eHEALS total sum scores between the online and paper samples (P=.12), the analysis was conducted using the total sample (see Table 3).

Table 3. Descriptive statistics of the individual eHealth Literacy Scale (eHEALS) items and total sum score for the online, paper, and total samples.
eHEALS itemOnline (n=388)Paper (n=315)Total sample (n=703)

MedianSkewMedianSkewMedianMean (SD)P value
Item 14–1.024–0.76
3.7 (1.1)
Item 24–0.914–0.9543.7 (1.0)N/A
Item 33–0.554–0.5543.4 (1.0)N/A
Item 44–0.574–0.6143.5 (1.0)N/A
Item 54–0.874–0.9543.7 (0.9)N/A
Item 64–1.404–1.2344.2 (0.9)N/A
Item 74–0.774–0.8143.9 (0.9)N/A
Item 83–0.243–0.2133.0 (1.1)N/A
eHEALS total sum score30–0.6729–0.763028.5 (6.2)0.12a (0.12b)

aMedian test.

bWilcoxon rank-sum test.

Classical Test Theory Analysis

Given the non-significant difference in the eHEALS total sum scores between theoretically distinct sampling frames (paper and online collection methods), a series of exploratory factor analyses were conducted to examine the strength of the general factor across the samples. As shown in Figure 2, the first eigenvalues indicated the presence of a strong general factor across sampling frames as well as for the total combined sample. Specifically, the first and second eigenvalue ratios across the samples were 4.62 and 1.07 = 4.32 (online), 4.54 and 1.15 = 3.95 (paper), and 4.57 and 1.09 = 4.19 (total). The large eigenvalue ratios suggest negligible multidimensionality across sample frames [33]. This was supported by omega reliability estimates across the samples of 0.90 (online), 0.89 (paper), and 0.89 (total) [29].

When considered collectively with the non-significant difference in eHEALS total scores between sampling frames, we used IRT to test the unidimensionality assumptions with the total eHEALS sample.

Figure 2. Exploratory factor scree plots to examine the strength of the general factors across the online, paper, and total samples.
View this figure

Item Response Theory Analysis

Separate bifactor models with two group factors and three group factors were computed. A direct model comparison between the bifactors indicated the two-group factor model exhibited significantly greater fit than the three-group factor model (χ21=58.4, P<.001).

Furthermore, the item loadings between the unidimensional and bifactor models were compared to determine the impact on the bias from ignoring suspected multidimensionality (literally, comparing across models with and without additional dimensions). Table 4 lists the loadings across the eHEALS unidimensional and bifactor models.

The average relative parameter bias (0.09; Table 5) value indicates that the impact of ignoring the multidimensionality of eHEALS by using unidimensional scores was negligible [34].

Table 4. Item response theory electronic health item loadings, N=703.
ItemUnidimensional modelBifactor model with two group factors

General FactorFactor 1Factor 2
Table 5. Item response theory electronic health unidimensionality indices, N=703.
Unidimensionality indexValue
Omega reliability0.99
Hierarchical omega0.92
H replicability0.95
Factor determinacy0.99
IECVc (number of items >0.80)5

aECV: estimated common variance.

bARPB: average relative parameter bias.

cIECV: item estimated common variance.

To verify this inference, first, we examined the correlation between the eHEALS and a meaningful substantive variable from the survey with the parents (sum score on how often parents use several digital media for general child health and development), which was significant in the expected direction (r=0.29, P<.01). Second, we examined the difference in this correlation between the unidimensional and bifactor scores of the eHEALS and digital media use for general child health and development. The results indicated observably small changes in the correlation (rΔ=0.02), which was tested and was not significant (z(1)=0.37, P=.71) for the eHEALS total sum score correlation with this substantive variable.

The findings of this study support the usefulness of the eHEALS measure as a unidimensional index for further studies. Specifically, we found a strong general factor of the eHEALS across distinct sampling frames as well as adequate reliability. Furthermore, the IRT analyses indicated minimal distortion of the primary factor from ignoring potential multidimensionality, and subscale reliabilities were inadequate to recommend further usage.

With respect to the EFA, this study used a different methodology to add to the current discussion of the eHEALS factor structure. Norman and Skinner [9] and other researchers factorizing the eHEALS [17,35-37] used principal component analysis for data reduction, while we implemented factor analysis to identify underlying latent constructs [38]. Another issue with previous analyses of the eHEALS is that the Likert scales were treated as continuous variables. In our analyses, we considered the ordinal structure of the eHEALS items using a polychoric correlation matrix when performing the EFA. Our results show the predominance of a single factor across both paper and online samples. Importantly, we also found no significant difference in the eHEALS total sum scores between the paper and online sampling frames. Unidimensionality assumptions were further tested using IRT analyses.

Given the construction and theorized application of eHEALS as a broad construct, our IRT analyses included a bifactor model. Consistent with previous IRT analyses, our findings indicated unidimensionality of the eHEALS [9,17,39]. Our findings also agree with the sampling-measurement interrelationship, such that our difficulty parameters were lower in our relatively young sample than the average. Further evidence from the comparison of the bifactor modeling of the primary factor suggested that the distortion from ignoring the potential multidimensionality in our multimodal sample was negligible. Furthermore, the empirical reliability after accounting for the general factor was inadequate to support future subscale use. Our results are not directly comparable to those from the original study by Soellner et al [16] in a German sample, since their results were not based on the IRT methodology. There is some agreement between the findings, given that two group factors were a better fit for the bifactor model than three group factors. These findings suggest two future applications for the German eHEALS. First, researchers may wish to pilot new items to expand subscales to achieve sufficient reliability. Second, researchers may wish to use a subset of items for purely unidimensional purposes. In the latter case, we refer readers to a potential core subset that could be comprised of the 5 items in Table 4 with item estimated common variances >.80. These are reported in Multimedia Appendix 2 with a preliminary differential item functioning analysis that indicated no bias of the items across paper and online samples.

In summary, the results support the broad but unidimensional factor structure of the German eHEALS. Our ordinal factor analysis supports the presence of a strong general factor. Furthermore, the item response theory analysis using bifactor models with one general factor and two or three group factors showed that the model with two group factors fitted better than the one with three factors. Comparing this bifactor model with two group factors with the unidimensional loadings did not suggest a substantial difference in primary loadings. Finally, we found no support for using eHEALS subscales.

The use of subscales in previous research [16,20,40] may underpin the recommendation to have 3 to 5 items per common factor. Although an advantage of the eHEALS might be its short length (only 8 items) from a methodological viewpoint, the factors may be underdetermined if the items were split into subscales [38]. Still, the benefit of different subdimensions of eHealth literacy would lie in the ability to identify possible areas of intervention. Conceptual models on general health literacy include the components of understanding, appraising, and applying health information [41]. For example, item 8 of the eHEALS “I feel confident in using information from the internet to make health decisions” could reflect the dimension of applying health information. As others have already indicated [22], to add further items to help discriminate components of eHealth literacy, we require a better understanding of the concepts study participants associate with particular eHEALS items. However, given that past findings have indicated a mixed number of factors underlying the eHEALS, researchers should carefully consider the overspecification value relative to the general stability. In this study, the unidimensional bifactor model of the eHEALS was stable across the two distinct samples (online and paper modes).


This study has some limitations. Although the parents were asked about their own eHealth literacy, it is likely that the questions on child health prompted the parents to answer the eHEALS items from the perspective of child health. This would explain the parents’ reluctance to make decisions based on internet-based health information (item 8 of the eHEALS). Therefore, in comparison with studies on adult eHealth literacy, parental eHealth literacy might be lower. However, the high information needs of parents, especially right after birth, might have increased eHealth literacy simply through practice and experience. Regarding the sample characteristics, the generalizability of our findings might be limited by the fairly low response rate (769/842, 30%) and the uniquely high socioeconomic status.

Another limitation is that measurement invariance was not assessed in terms of the participants’ individual characteristics. For example, future studies could focus specifically on the generalizability by gender of our proposed unidimensional eHEALS. This was not the focus of the current study and will be addressed in further analyses. It is important, furthermore, for future researchers to consider the relevance of their samples when studying eHEALS measurement properties. This study aimed to extend the application of the eHEALS among new parents.


This study suggests that the German eHEALS possesses a broad, unidimensional factor structure among Swiss-German parents. Although the two samples differed with respect to participant characteristics such as age, education, and income, we failed to find a significant difference in the eHEALS total sum scores. The underrepresentation of participants of lower socioeconomic status, not only in our study but also in many other studies on digital health, warrants future studies to over-sample this population. We found similar factor structures and item properties irrespective of application mode. That is, the EFAs suggested a strong general factor. Finally, bifactor modeling did not outperform the unidimensional model, and subscales were unsupported because of low reliability. While using the total sum score is appropriate to assess eHealth literacy, further development and refinement of the eHEALS are proposed to address specific sub-domains of eHealth literacy. For any sample, practitioners should use only the eHEALS total score, and future research aiming to utilize subscales should expand the eHEALS item pool for empirical testing.


The authors thank the municipalities who provided the parents’ addresses, all participating parents for their valuable engagement, and our colleague Dominik Robin for his support in study design. This study was funded by the Käthe Zingg-Schwichtenberg Fund (KZS) of the Swiss Academy of Medical Sciences (SAMS). The authors of this manuscript are independent of the funding agencies. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Detailed protocol of the exploratory factor analysis with polychoric correlation matrix.

DOCX File , 28 KB

Multimedia Appendix 2

Potential core set of the German eHEALS and differential item functioning analysis (DIF).

DOCX File , 20 KB

  1. Plantin L, Daneback K. Parenthood, information and support on the internet. A literature review of research on parents and professionals online. BMC Fam Pract 2009 May 18;10:34 [FREE Full text] [CrossRef] [Medline]
  2. Skranes LP, Løhaugen GCC, Botngård A, Skranes J. Internet use among mothers of young children in Norway—a survey of Internet habits and perceived parental competence when caring for a sick child. J Public Health 2014 Jul 2;22(5):423-431. [CrossRef]
  3. Tuffrey C, Finlay F. Use of the internet by parents of paediatric outpatients. Arch Dis Child 2002 Dec;87(6):534-536 [FREE Full text] [CrossRef] [Medline]
  4. Wainstein BK, Sterling-Levis K, Baker SA, Taitz J, Brydon M. Use of the Internet by parents of paediatric patients. J Paediatr Child Health 2006 Sep;42(9):528-532. [CrossRef] [Medline]
  5. Scullard P, Peacock C, Davies P. Googling children's health: reliability of medical advice on the internet. Arch Dis Child 2010 Aug;95(8):580-582. [CrossRef] [Medline]
  6. Barkin JL, Jani S. Information Management in New Motherhood: Does the Internet Help or Hinder? J Am Psychiatr Nurses Assoc 2016 Nov;22(6):475-482. [CrossRef] [Medline]
  7. DeLuca JM, Kearney MH, Norton SA, Arnold GL. Internet use by parents of infants with positive newborn screens. J Inherit Metab Dis 2012 Sep;35(5):879-884. [CrossRef] [Medline]
  8. van der Gugten AC, de Leeuw RJRJ, Verheij TJM, van der Ent CK, Kars MC. E-health and health care behaviour of parents of young children: a qualitative study. Scand J Prim Health Care 2016 Jun;34(2):135-142 [FREE Full text] [CrossRef] [Medline]
  9. Norman CD, Skinner HA. eHEALS: The eHealth Literacy Scale. J Med Internet Res 2006 Nov 14;8(4):e27 [FREE Full text] [CrossRef] [Medline]
  10. Knapp C, Madden V, Wang H, Sloyer P, Shenkman E. Internet use and eHealth literacy of low-income parents whose children have special health care needs. J Med Internet Res 2011 Sep 29;13(3):e75 [FREE Full text] [CrossRef] [Medline]
  11. Knapp C, Madden V, Marcu M, Wang H, Curtis C, Sloyer P, et al. Information seeking behaviors of parents whose children have life-threatening illnesses. Pediatr Blood Cancer 2011 May;56(5):805-811. [CrossRef] [Medline]
  12. Norman CD, Skinner HA. eHealth Literacy: Essential Skills for Consumer Health in a Networked World. J Med Internet Res 2006 Jun 16;8(2):e9 [FREE Full text] [CrossRef] [Medline]
  13. Karnoe A, Kayser L. How is eHealth literacy measured and what do the measurements tell us? A systematic review. Knowledge Management & E-Learning 2015;7(4):576-600. [CrossRef]
  14. van der Vaart R, van Deursen AJ, Drossaert CH, Taal E, van Dijk JA, van de Laar MA. Does the eHealth Literacy Scale (eHEALS) measure what it intends to measure? Validation of a Dutch version of the eHEALS in two adult populations. J Med Internet Res 2011 Nov 09;13(4):e86 [FREE Full text] [CrossRef] [Medline]
  15. Neter E, Brainin E. Perceived and Performed eHealth Literacy: Survey and Simulated Performance Test. JMIR Hum Factors 2017 Jan 17;4(1):e2 [FREE Full text] [CrossRef] [Medline]
  16. Soellner R, Huber S, Reder M. The Concept of eHealth Literacy and Its Measurement. Journal of Media Psychology 2014 Jan;26(1):29-38. [CrossRef]
  17. Diviani N, Dima AL, Schulz PJ. A Psychometric Analysis of the Italian Version of the eHealth Literacy Scale Using Item Response and Classical Test Theory Methods. J Med Internet Res 2017 Apr 11;19(4):e114 [FREE Full text] [CrossRef] [Medline]
  18. Richtering SS, Morris R, Soh S, Barker A, Bampi F, Neubeck L, et al. Examination of an eHealth literacy scale and a health literacy scale in a population with moderate to high cardiovascular risk: Rasch analyses. PLoS One 2017;12(4):e0175372 [FREE Full text] [CrossRef] [Medline]
  19. Neter E, Brainin E, Baron-Epel O. The dimensionality of health literacy and eHealth literacy. European Health Psychologist 2015;17(6):275-280.
  20. Hyde LL, Boyes AW, Evans T, Mackenzie LJ, Sanson-Fisher R. Three-Factor Structure of the eHealth Literacy Scale Among Magnetic Resonance Imaging and Computed Tomography Outpatients: A Confirmatory Factor Analysis. JMIR Hum Factors 2018 Feb 19;5(1):e6 [FREE Full text] [CrossRef] [Medline]
  21. Sudbury-Riley L, FitzPatrick M, Schulz PJ. Exploring the Measurement Properties of the eHealth Literacy Scale (eHEALS) Among Baby Boomers: A Multinational Test of Measurement Invariance. J Med Internet Res 2017 Feb 27;19(2):e53 [FREE Full text] [CrossRef] [Medline]
  22. Stellefson M, Paige SR, Tennant B, Alber JM, Chaney BH, Chaney D, et al. Reliability and Validity of the Telephone-Based eHealth Literacy Scale Among Older Adults: Cross-Sectional Survey. J Med Internet Res 2017 Dec 26;19(10):e362 [FREE Full text] [CrossRef] [Medline]
  23. Kerry MJ, Wang R, Bai J. Assessment of the Readiness for Interprofessional Learning Scale (RIPLS): An item response theory analysis. J Interprof Care 2018 Sep;32(5):634-637. [CrossRef] [Medline]
  24. Jaks R, Baumann I, Juvalta S, Dratva J. Parental digital health information seeking behavior in Switzerland: a cross-sectional study. BMC Public Health 2019 Feb 21;19(1):225-235. [CrossRef]
  25. Hollander M, Wolfe D, Chicken E. Nonparametric Statistical Methods. In: John Wiley & Sons. Hoboken, New Jersey. USA: John Wiley & Sons, Inc; 2013:978-971.
  26. Holgado–Tello FP, Chacón–Moscoso S, Barbero–García I, Vila–Abad E. Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Qual Quant 2008 Sep 7;44(1):153-166. [CrossRef] [Medline]
  27. Revelle W. CRAN.R-project. psych: Procedures for Personality and Psychological Research. Northwestern University, Evanston, Illinois, USA; 2018. Procedures for Personality and Psychological Research   URL:
  28. O’Connor BP. SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers 2000 Sep;32(3):396-402. [CrossRef]
  29. McDonald R. Test theory: A unified treatment. Mahwah, NJ. US: Lawrence Erlbaum Associates Publishers; 1999.
  30. Reise SP, Morizot J, Hays RD. The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Qual Life Res 2007 May 4;16(S1):19-31. [CrossRef] [Medline]
  31. DeMars CE. A Tutorial on Interpreting Bifactor Model Scores. International Journal of Testing 2013 Oct;13(4):354-378. [CrossRef] [Medline]
  32. Dueber D. Educational, School, and Counseling Psychology Research Tools. 2017. Bifactor Indices Calculator: A Microsoft Excel-Based Tool to Calculate Various Indices Relevant to Bifactor CFA Models   URL:
  33. Embretson S, Reise S. Item Response Theory for Psychologists. Mahwah, New Jersey: Psychology Press; 2000.
  34. Muthén B, Kaplan D, Hollis M. On structural equation modeling with data that are not missing completely at random. Psychometrika 1987 Sep;52(3):431-462. [CrossRef]
  35. Paramio Pérez G, Almagro BJ, Hernando Gómez Á, Aguaded Gómez JI. [Validation of the eHealth Literacy Scale (eHEALS) in Spanish University Students]. Rev Esp Salud Publica 2015 Jun;89(3):329-338 [FREE Full text] [CrossRef] [Medline]
  36. Mitsutake S, Shibata A, Ishii K, Okazaki K, Oka K. Developing Japanese version of the eHealth Literacy Scale (eHEALS). Nihon Koshu Eisei Zasshi 2011 May;58? 2011:71.
  37. Del Giudice P, Bravo G, Poletto M, De Odorico A, Conte A, Brunelli L, et al. Correlation Between eHealth Literacy and Health Literacy Using the eHealth Literacy Scale and Real-Life Experiences in the Health Sector as a Proxy Measure of Functional Health Literacy: Cross-Sectional Web-Based Survey. J Med Internet Res 2018 Oct 31;20(10):e281 [FREE Full text] [CrossRef] [Medline]
  38. Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ. Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods 1999 Sep;4(3):272-299. [CrossRef]
  39. Nguyen J, Moorhouse M, Curbow B, Christie J, Walsh-Childers K, Islam S. Construct Validity of the eHealth Literacy Scale (eHEALS) Among Two Adult Populations: A Rasch Analysis. JMIR Public Health Surveill 2016 May 20;2(1):e24 [FREE Full text] [CrossRef] [Medline]
  40. Paige SR, Miller MD, Krieger JL, Stellefson M, Cheong J. Electronic Health Literacy Across the Lifespan: Measurement Invariance Study. J Med Internet Res 2018 Jul 09;20(7):e10434. [CrossRef]
  41. Sørensen K, Van den Broucke S, Fullam J, Doyle G, Pelikan J, Slonska Z, (HLS-EU) Consortium Health Literacy Project European. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health 2012 Jan 25;12:80 [FREE Full text] [CrossRef] [Medline]

EFA: exploratory factor analysis.
eHEALS: eHealth Literacy Scale.
IRT: item response theory.

Edited by G Eysenbach; submitted 16.07.19; peer-reviewed by E Neter, M Fiordelli; comments to author 06.09.19; revised version received 22.10.19; accepted 12.11.19; published 13.03.20


©Sibylle Juvalta, Matthew J Kerry, Rebecca Jaks, Isabel Baumann, Julia Dratva. Originally published in the Journal of Medical Internet Research (, 13.03.2020.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.