Parents increasingly use digital sources when seeking information on their child's health [1-3]. Through their accessibility, digital sources offer the opportunity for parents to feel empowered [4] (eg, to verify information received from health professionals), consider alternative treatment options, and develop communal networks with other families and patients with a common disease or condition. However, because the quality and reliability of information from digital sources vary substantially [5], the information can be overwhelming and cause insecurity or anxiety [6-8]. Therefore, eHealth literacy of parents is critical to maximize the potential benefits of digital media for children’s health. eHealth literacy has been defined as “a set of skills required to effectively engage information technology for health” [9].

Research on the eHealth literacy of parents is lacking. A study by Knapp et al [10] in Florida showed that low-income parents of children with special health needs had high levels of internet use for information purposes. However, half of the study participants had difficulties separating high-quality from low-quality information and were not confident using the internet. Similar findings were found in a study of parents whose children had life-threatening illnesses [11]. Both studies used the eHealth Literacy Scale (eHEALS) [9]. Both studies are also older, considering the fast-evolving context of internet acculturation. The eHEALS was developed by Norman and Skinner [12], who defined eHealth literacy as “the ability to seek, find, understand, and appraise health information from electronic sources and apply the knowledge gained to addressing or solving a health problem.” In this regard, the eHEALS pertains to the critical consumption of extant internet content rather than the creation of new content. The eHEALS is a widely applied instrument that has been validated internationally in diverse languages [13]. Adequate internal consistency has been found [13]. However, van der Vaart et al [14] reported weak correlations between the eHEALS scores and tasks on an eHealth performance test. In contrast, more recent research found significant associations between perceptions and performance [15]. In addition, inconsistent results for the factor structure of the eHEALS have been reported. Norman and Skinner [9] proposed a one-factor structure for the original English eHEALS. For the German eHEALS, Soellner et al [16] determined a two-factor structure using confirmatory factor analysis. For an Italian version of the eHEALS applied in the Italian-speaking area of Switzerland, the researchers [17] recommended using the eHEALS total sum score after applying Rasch (item response theory [IRT]) modeling. However, a study with patients at risk of cardiovascular disease in Australia also applied Rasch modeling and concluded that the eHEALS captures different aspects of eHealth literacy, which may have to be scored separately [18]. Neter et al [19] reported a different two-factor solution for adults in Israel aged at least 21 years than the two-factor solution used by Soellner et al [16]. Moreover, two recent studies developed a three-factor solution for the English eHEALS: awareness, skills, and evaluation [20,21]. A three-factor solution was also reported in an IRT analysis of eHEALS, although the authors noted that substantially high interfactor correlations “support an overarching structure of eHEALS” [22]. Notably, both studies implementing IRT analyses of the eHEALS found measurement properties (item difficulties) reflective of their study samples. That is, Diviani et al [17] found wide variability in item difficulties in a broad sample of people aged 16-71 years, and Stellefson et al [22] found high item difficulties in a sample narrowed to older adults.

The eHEALS was originally constructed for broad usage, as creators Norman and Skinner [9] state, “this article describes the development and psychometric evaluation of a measure of eHealth literacy designed for broad use in supporting consumer eHealth in public health and clinical care” (p2). Subsequent research has reported unstable latent factor structures underlying the eHEALS measure, despite the conventional understanding that “broader constructs are stabilized with broad factors” [23]. Given the inconsistent results for the eHEALS, we aimed to establish the psychometric structure of the eHEALS in parents participating in the Digital Parental Counselors study (in German: Digitale Elternratgeber), which investigated digital media use by parents for their children’s health in the German-speaking area of Switzerland [24]. Specifically, we aimed to explore the factor structure of the German eHEALS and to assess the viability of subscales using IRT and bifactor modeling. We also addressed the methodological issues concerning the handling of Likert scales and the use of factor analysis experienced with previous research of the eHEALS.

The findings of this study support the usefulness of the eHEALS measure as a unidimensional index for further studies. Specifically, we found a strong general factor of the eHEALS across distinct sampling frames as well as adequate reliability. Furthermore, the IRT analyses indicated minimal distortion of the primary factor from ignoring potential multidimensionality, and subscale reliabilities were inadequate to recommend further usage.

With respect to the EFA, this study used a different methodology to add to the current discussion of the eHEALS factor structure. Norman and Skinner [9] and other researchers factorizing the eHEALS [17,35-37] used principal component analysis for data reduction, while we implemented factor analysis to identify underlying latent constructs [38]. Another issue with previous analyses of the eHEALS is that the Likert scales were treated as continuous variables. In our analyses, we considered the ordinal structure of the eHEALS items using a polychoric correlation matrix when performing the EFA. Our results show the predominance of a single factor across both paper and online samples. Importantly, we also found no significant difference in the eHEALS total sum scores between the paper and online sampling frames. Unidimensionality assumptions were further tested using IRT analyses.

Given the construction and theorized application of eHEALS as a broad construct, our IRT analyses included a bifactor model. Consistent with previous IRT analyses, our findings indicated unidimensionality of the eHEALS [9,17,39]. Our findings also agree with the sampling-measurement interrelationship, such that our difficulty parameters were lower in our relatively young sample than the average. Further evidence from the comparison of the bifactor modeling of the primary factor suggested that the distortion from ignoring the potential multidimensionality in our multimodal sample was negligible. Furthermore, the empirical reliability after accounting for the general factor was inadequate to support future subscale use. Our results are not directly comparable to those from the original study by Soellner et al [16] in a German sample, since their results were not based on the IRT methodology. There is some agreement between the findings, given that two group factors were a better fit for the bifactor model than three group factors. These findings suggest two future applications for the German eHEALS. First, researchers may wish to pilot new items to expand subscales to achieve sufficient reliability. Second, researchers may wish to use a subset of items for purely unidimensional purposes. In the latter case, we refer readers to a potential core subset that could be comprised of the 5 items in Table 4 with item estimated common variances >.80. These are reported in Multimedia Appendix 2 with a preliminary differential item functioning analysis that indicated no bias of the items across paper and online samples.

In summary, the results support the broad but unidimensional factor structure of the German eHEALS. Our ordinal factor analysis supports the presence of a strong general factor. Furthermore, the item response theory analysis using bifactor models with one general factor and two or three group factors showed that the model with two group factors fitted better than the one with three factors. Comparing this bifactor model with two group factors with the unidimensional loadings did not suggest a substantial difference in primary loadings. Finally, we found no support for using eHEALS subscales.

The use of subscales in previous research [16,20,40] may underpin the recommendation to have 3 to 5 items per common factor. Although an advantage of the eHEALS might be its short length (only 8 items) from a methodological viewpoint, the factors may be underdetermined if the items were split into subscales [38]. Still, the benefit of different subdimensions of eHealth literacy would lie in the ability to identify possible areas of intervention. Conceptual models on general health literacy include the components of understanding, appraising, and applying health information [41]. For example, item 8 of the eHEALS “I feel confident in using information from the internet to make health decisions” could reflect the dimension of applying health information. As others have already indicated [22], to add further items to help discriminate components of eHealth literacy, we require a better understanding of the concepts study participants associate with particular eHEALS items. However, given that past findings have indicated a mixed number of factors underlying the eHEALS, researchers should carefully consider the overspecification value relative to the general stability. In this study, the unidimensional bifactor model of the eHEALS was stable across the two distinct samples (online and paper modes).