The internet is used by many individuals in the context of health. In a 2024 survey conducted in the European Union, 58% of the participants aged between 16 and 74 years reported that they had used the internet to search for health-related information within the last 3 months [1]. The internet offers more than access to health-related information. Today, a variety of digital tools offer further use cases in the context of health. For example, users can generate health information via generative artificial intelligence (GenAI)–based chatbots [2] and share health information or interact with others via social media platforms [3]. In addition, dedicated health applications such as Apple Health [4] and Google Fit [5] help individuals track health goals via smartphones and smart watches. The World Health Organization provides an overview of digital interventions for individuals in the context of health that enable communication, tracking, reporting, financial transactions, and consent management [6].

However, digital tools are also associated with obstacles for individuals who lack access to hardware and internet or digital literacy [7]. Subgroups, such as older adults, are particularly at risk of digital exclusion [8,9]. Furthermore, users are confronted with misinformation on health topics. A review of studies that analyzed health-related posts on social media platforms such as Facebook, Instagram, and Twitter, now called X, indicates that health misinformation is widespread [10]. Similarly, studies show that GenAI models generate unreliable health information in some cases [11,12]. Users who follow incorrect advice generated by ChatGPT may be exposed to serious health risks [12].

As often argued in the literature, sufficient eHealth literacy (eHL) is required for the reliable use of digital tools in the context of health [13-16]. Norman and Skinner [13] proposed an early and influential definition of eHL in 2006, long before the GenAI era. They defined eHL as “the ability to seek, find, understand, and appraise health information from electronic sources and apply the knowledge gained to addressing or solving a health problem” ([13], p. 1). Information on eHL levels is relevant for providers of digital interventions to develop targeted solutions or physicians to suggest appropriate digital health interventions and to evaluate the level of support required by individual patients [17].

Different instruments have been developed to measure eHL and used to varying degrees in empirical research. The most common eHL measurement instrument is the eHealth Literacy Scale (eHEALS) [18-20]. The eHEALS is an 8-item self-report measure developed by Norman and Skinner [17] and published in 2006. It was initially tested among students aged 13 to 21 years [17] and has been used in a variety of studies, involving internet users and the general population. However, researchers pointed out limitations of the eHEALS, including its one-dimensionality [21,22], insufficient capture of actual eHL levels due to the risk of overestimation and underestimation [14,23], and lack of items related to shifting use cases, such as social media [14,24]. Researchers also argued that the underlying Lily model, named after its visual arrangement of 6 literacies around eHL—traditional literacy, information literacy, media literacy, computer literacy, scientific literacy, and health literacy [13,17]—does not cover important aspects of eHL, such as the ability to formulate health questions; know-how in using digital technologies; and contextual, cultural, and communicative competencies [15] or individual, situational, and environmental factors [25]. Another aspect missing in the Lily model is mentioned in the eHealth Literacy Framework (eHLF) by Norgaard et al ([26], p. 533): “safety and control,” particularly related to personal health data. In light of the conceptual discourse and shifting use cases, further eHL measurement instruments have been developed. Prominent examples are the multidimensional eHealth Literacy Questionnaire (eHLQ) [27], which is based on the eHLF [26], and the Digital Health Literacy Instrument (DHLI), which covers skills related to social media [14]. Similar to the eHEALS [17], these instruments provide eHL scores based on self-reports [14,27]. However, the DHLI is at least supplemented by a set of performance-based items [14]. A review indicates that performance-based approaches are less common in empirical research [18]. In March 2025, Norman, Skinner, and colleagues [28] published an updated definition of eHL, which takes “effectiveness, safety and helpfulness” into account and is as follows: “the ability to engage with digital technologies in effective, safe, and helpful ways to achieve health goals” ([28], p. 1). They also updated the Lily model, which now considers contextual factors surrounding eHL, and developed a revised 10-item version of the eHEALS based on the new Lily model, which has not yet been published [28].

Overall, the outlined historical development shows that the definition and operationalization of eHL has changed since the initial definition by Norman and Skinner [13] was published in 2006. Measuring eHL in a fast-paced digital environment is a challenging task. Researchers and health care providers not only need suitable eHL measurement instruments but also need to select appropriate data collection methods and modes, especially if they aim to assess the eHL of vulnerable subgroups. For example, the use of online surveys can lead to an underrepresentation of non-internet users [29]. To provide guidance for researchers, practitioners, and policymakers, there is a need for an up-to-date overview of current eHL measurement instruments, data collection methods and modes, and associated limitations. The latter is particularly important, as existing reviews in the field of eHL measurement [eg, 18-20] did not systematically analyze eHL measurement–related limitations reported in the included papers.

This paper has two central objectives: (1) to provide an overview of eHL measurement instruments as well as data collection methods and modes recently used in empirical research in the European Economic Area, Switzerland, and the United Kingdom (ie, in countries that are well comparable in terms of health care system standards, culture, socioeconomic aspects, and data protection) and (2) to examine the limitations of the eHL measurement instruments and data collection approaches reported in the included papers. In addition, it aims to analyze the overall publication trend, eHL assessment frequency per participant, countries of data collection, geographical distribution of eHL measurement instruments, data collection languages, target groups, and sample sizes.

The following research question served as the basis for this paper: how has eHL been assessed in the European Economic Area, Switzerland, and the United Kingdom in recent years, and which methodological limitations need to be considered?

The study protocol was published on PROSPERO on February 07, 2025 (CRD42025642890). It was updated on November 06, 2025, as the scope and methods—including the eligibility criteria to reduce the number of relevant records—were refined, and again on March 05, 2026, following the updated searches conducted on January 28, 2026. As can be seen in the protocol, the study was initially planned as a rapid review but evolved into a scoping review, which maps existing eHL measurement approaches and associated limitations.

Multimedia Appendix 1 provides an overview of the inclusion and exclusion criteria developed by TSB and JVW in consultation with JPE and JN. Papers that described empirical studies and were published in peer-reviewed academic journals were included. Papers had to be published in 2020 or later for two reasons: (1) rapid technological advances and associated new use cases in the context of health, for example, in the field of GenAI [2] and (2) regulatory changes. For example, Germany introduced the Digital Care Act (Digitale-Versorgung-Gesetz [DVG]) in 2019, which has regulated and enabled the reimbursement of prescribable digital health applications (Digitale Gesundheitsanwendungen [DiGA]) [31]. Regulatory measures regarding the regulation and reimbursement of digital health interventions have been taken in other European countries as well [32,33]. In addition, papers were included only if abstracts and full texts were available in either English or German. Papers had to describe studies with a clear geographic focus on the European Economic Area, Switzerland, and the United Kingdom. These countries are well comparable in terms of culture and socioeconomic aspects and have developed health care systems. Furthermore, data protection is strictly regulated: in the European Economic Area by the General Data Protection Regulation (GDPR) [34], in Switzerland by the Federal Act on Data Protection (FADP) [35], and in the United Kingdom by the UK GDPR and the Data Protection Act 2018 [36]. Papers had to report the use of all items or subscales of an original eHL measurement instrument intended for calculating eHL scores.

Papers were excluded if they were reviews. Study protocols, conference papers, gray literature, and book chapters were also excluded. Moreover, cross-country studies without a clear focus on the target regions were excluded. In addition, papers were excluded if they reported the use of a measurement instrument not capturing the skills described in Norman and Skinner’s [13] 2006 definition of eHL, the use of a measurement instrument solely for eHL subdimensions(eg, traditional literacy, health literacy, information literacy, scientific literacy, media literacy, and computer literacy [13]), or the use of a context-specific eHL measurement instrument (eg, COVID-19 or health conditions). Furthermore, studies that had a clear focus on adolescents or children but not on adults were excluded. Studies involving individuals with a professional perspective (eg, health professionals, IT specialists, teachers, and trainers, as well as students in health, IT, education, or communication study programs) were also excluded given the focus of this paper on eHL measurement instruments that are suitable for the general population, patients, parents, or informal caregivers.

It should be noted that it was not initially planned to exclude papers without a specific geographic focus on the European Economic Area, Switzerland, and the United Kingdom. However, after the title-abstract screening of the records from the initial searches, many records from a variety of countries with differences in culture, socioeconomic conditions, data protection regulation, and health care quality remained. To improve the comparability and facilitate the synthesis, the geographic focus eligibility criterion was added. The search strategy itself would not have changed if this criterion had been defined from the beginning.

The reporting of the search process adheres to the PRISMA-S (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Literature Search Extension) checklist [37].

The screening of the records identified via the initial searches and updated searches was carried out separately at 2 different points in time. The method was identical except for one difference. The geographical focus eligibility criterion was applied only during the full-text screening of the records identified via the initial searches and not during the title-abstract screening. During the screening of the records identified via the updated searches, this criterion was applied from the beginning.

In both cases, EndNote was used for automatic deduplication. Further duplicates were later removed manually. ASReview, an open-source machine learning tool, which ranks records based on the estimated relevance, was seen as a solution to handle the high number of records identified [40]. Van de Schoot et al ([40], p. 125) showed that ASReview “can yield far more efficient reviewing than manual reviewing while providing high quality.” The records were divided by language because ASReview may have difficulty processing different languages: (1) English titles and abstracts and (2) German titles or abstracts. As most records were in English, only records with English titles and abstracts were screened in ASReview, while the smaller number of records with German titles or abstracts were screened manually.

To train the ASReview model, an initial training set containing relevant and irrelevant records was required [40]. JVW imported all English records into Rayyan, a digital tool for collaborative screening [41], created a 5% sample using the sampling function, screened the titles and abstracts and identified relevant records: 33 from the sample of the initial searches, and 2 from the sample of the updated searches. To reduce potential bias in the training set, the same number of irrelevant records was randomly selected. TSB and JN reviewed and confirmed both sets of relevant and irrelevant records. The English records were imported into ASReview, and the training set was used to train the ASReview model. ASReview default settings were used (ie, feature extraction technique: term frequency-inverse document frequency (TF-IDF), classifier: naïve Bayes, query strategy: maximum, and balance strategy: dynamic resampling/double). JVW conducted the title-abstract screening in ASReview. As recommended, the records were screened until a predefined stopping criterion was reached [40]. In line with previous review processes [42-44], the ASReview stopping criterion was set at 100 consecutive irrelevant records. The remaining English records were excluded.

Records with German titles or abstracts identified via both searches were manually screened in Rayyan. In both cases, a 25% sample was selected using the sampling function. TSB and JVW independently screened the records in both samples. Conflicts were resolved through discussion between the two authors. JVW screened the remaining records.

English and German full-text reports were retrieved and imported into Rayyan. In both cases, a 25% sample was created using the sampling function; TSB and JVW independently screened these records. Most conflicts were resolved through discussion between both authors. Some conflicts were discussed in the whole research team. JVW screened the remaining full-text reports and consulted TSB in cases of uncertainty to ensure a consistent and accurate selection.

All authors developed a standardized extraction form, tested it in advance, and modified it on the basis of a sample of 5 included papers [39] and reviewer feedback. JVW extracted data from the included papers using MAXQDA (version 24; VERBI Software Consult Sozialforschung GmbH) to manually code relevant text passages according to predefined categories. TSB then verified the correctness of all extracted data.

The final extraction form contained the following deductive categories: (1) author and publication year, (2) title, (3) full-text language, (4) main objective, (5) objective to validate an eHL measurement instrument, (6) data collection period, (7) eHL measurement frequency per participant, (8) (target) age groups, (9) specific participants’ perspectives, (10) country focus, (11) data collection language, (12) sample size in the central eHL measurement part(s), (13) eHL measurement instrument, (14) data collection method and mode, and (15) reported limitations regarding the eHL measurement instrument and data collection method and modes. Additional subcategories were developed inductively based on the material. Information that was not explicitly reported in the included papers was inferred from the context where reasonable, as indicated in Multimedia Appendix 3.

The synthesis focused on the following eight themes and questions:

As the primary objective of this paper was to analyze how eHL was measured, the role of eHL as a variable in each included paper was not analyzed. Because the reporting was sometimes insufficiently transparent and traceable, it was not always possible to determine whether the included papers analyzed the same datasets as other included papers. Therefore, to ensure a consistent and objective approach, each included paper was considered in the synthesis, regardless of whether analyses were based on the same datasets.

Through database searches, 13,657 records were identified (CINAHL: n=3525; PubMed: n=10,132). Including the 400 records from Google Scholar, the total number of records was 14,057. Of these, 10,256 records were identified during the initial searches (January 31, 2025) and 3801 records during the updated searches (January 28, 2026). Of the records from the initial searches, 1944 duplicates were automatically removed using EndNote, resulting in 8312 unique records. Of the records from the updated searches, 509 duplicates were removed, resulting in 3292 unique records. Therefore, the total number of records to be potentially screened was 11,604. While all unique German records from the initial searches (113/113, 100%) and the updated searches (100/100, 100%) were manually screened in Rayyan, 2523 of the 8199 unique English records from the initial searches (30.77%) and 1115 of the 3192 unique English records from the updated searches (34.93%) were screened in ASReview until the predefined stopping criterion was reached. In total, 3076 irrelevant records and 7753 that remained after the ASReview stopping criterion was reached were excluded. The full texts of the remaining 775 records were retrieved and assessed for eligibility. For various reasons, 643 full-text reports were excluded. A total of 132 papers were included in the final analysis. Figure 1 shows the combined results of both selection points in time (initial and updated searches).

Some papers might appear to be eligible for inclusion but were excluded. For instance, some papers [eg, 45-48] reported only the use of selected subscales or items. As inclusion in the final analysis of this study required the reported use of all items or subscales of a version of an eHL measurement instrument intended for calculating eHL scores, these papers were excluded. Other examples of excluded papers include those that reported the use of context-specific versions of eHL measurement instruments (eg, related to COVID-19 [eg, 49-54] or chronic pain [55]).

Multimedia Appendix 3 presents the metadata extracted from the included papers [22,58-188].

All included papers (132/132, 100%) reported that data on eHL were collected via surveys. As shown in Table 2, most included papers (71/132, 53.79%) reported or implied that surveys were administered digitally (eg, online or on a tablet). Twenty-three of the 132 included papers (17.42%) reported or implied that 2 or 3 data collection modes were used. One of these (1/132, 0.76%) described a cross-country study in which different modes (1 or 2) were used depending on the country [117].

The review process used is subject to some limitations. First, it needs to be noted that, due to the use of ASReview to facilitate the title and abstract screening of the English records, 30.77% (2523/8199) of the English records from the initial searches and 34.93% (1115/3192) of the English records from the updated searches were screened. The remaining English records remained after the ASReview stopping criterion (ie, 100 consecutive irrelevant records) was reached in both cases. Thus, despite the demonstrated performance of ASReview [40], there is a risk that relevant papers were not screened and thus not included in the final analysis.

Second, the exclusion of papers without a clear focus on the target regions resulted in the omission of papers and eHL measurement instruments. As German and English papers were included, there may be an overrepresentation of papers from German- and English-speaking regions and an underrepresentation of papers from non–German-speaking and non–English-speaking regions within the target regions.

Third, the analysis covered limitations reported in the Limitations section or, if a Limitations section was unavailable, the Discussion section of the included papers. Therefore, limitations reported elsewhere in the included papers were not analyzed. Moreover, the analysis was limited to reported limitations regarding eHL measurement instruments as well as eHL-related data collection methods and modes. Other limitations reported in the included papers (eg, those related to cross-sectional study designs or sampling or recruitment strategies and limitations specific to translated and culturally adapted versions of eHL measurement instruments) were not extracted and analyzed in this study.

This paper provides health care practitioners and researchers with a structured overview of limitations related to eHL measurement instruments and data collection methods and modes. The identified eHL measurement instruments differ in key aspects, including length, complexity, and alignment with rapidly evolving digital use cases. Particularly, online surveys might lead to an exclusion of subgroups. Thus, it clearly shows that selecting both a suitable eHL measurement instrument and suitable data collection methods and modes is a key decision, which should be based on factors such as the purpose of the planned eHL measurement, the target population, and the data collection setting.

The overview could encourage researchers to adopt eHL measurements that are less common or were not used in specific countries yet. This potentially requires translation, cultural adaptation, and validation efforts. Future research should also focus on revising existing eHL measurement instruments and developing new eHL measurement instruments to capture the skills required for an ever-growing range of digital tools in the context of health, including social media and GenAI. This may require new underlying operationalization approaches that manage to avoid becoming obsolete. Use case–specific eHL measurement instruments may also be relevant for health care providers and researchers who only want to assess specific aspects (eg, skills that are necessary when using a specific digital health application) considering setting and time constraints (eg, in health care facilities). The fact that only self-report eHL measurement instruments were identified and the associated risk of self-report biases reported in the included papers indicate a need for performance-based eHL measurement approaches. Furthermore, revised or new eHL measurement instruments tailored to older adults are highly relevant for gaining a better understanding of the specific eHL-related needs.

Policymakers should encourage health care providers to conduct eHL measurements using carefully selected eHL measurement instruments and data collection methods and modes to provide targeted solutions based on individual eHL-related needs.

To our knowledge, this is the first scoping review synthesizing eHL measurement limitations reported in empirical papers from the European Economic Area, Switzerland, and the United Kingdom. The analysis of 132 empirical papers showed that eHL was assessed using 9 eHL measurement instruments in various languages, especially the eHEALS, and that digital surveys were a common data collection approach. The identified eHL measurement–related limitations, such as self-reported eHL measurement, exclusion of subgroups, and lack of alignment of technological developments and changing use cases in the context of health, potentially lead to biased results. Therefore, selecting suitable eHL measurement approaches is a critical decision. Health care professionals and researchers should consider a variety of factors, such as measurement purpose, target population, and data collection setting. There is a need for translated and culturally adapted as well as revised or newly developed eHL measurement instruments that keep pace with technological change, cover diverse use cases or focus on specific use cases, and contain performance-based components. Policymakers should encourage health care providers to thoughtfully select appropriate eHL measurement approaches and use eHL-related insights as a basis for targeted solutions.