In 2005, the World Health Organization defined eHealth as the dissemination of health resources and health care information through electronic means, enabling health care professionals and users to disseminate and access health information [1]. Based on the concept of electronic health, eHealth literacy was first proposed by Norman and Skinner [2], which refers to the ability to search, understand, and assess health information from electronic sources, as well as to deal with and apply the health information obtained, and eventually to solve health problems [2]. To date, this concept is most widely cited. However, with further research in the field and the increasing availability of electronic media, researchers’ perceptions of the connotation of eHealth literacy have changed. Unlike the concept of Norman and Skinner [2], which emphasizes individual capability to apply health information through electronic methods, subsequent studies on the concept of eHealth literacy are weighed on the interaction between the individual and technology and the individual and health technology service provider [3-7].

Entering the mobile age, electronic equipment gets broad applications, and various electronic health programs keep coming out, which enhance the accessibility of medical resources and participation of individuals in medical decisions, having a positive impact on mass health [1,8,9]. On the other hand, the diversity and varying quality of eHealth resources pose challenges for people with limited health literacy. They struggle to distinguish legitimate health services from fraudulent ones and identify credible information, resulting in low public acceptance and inefficient use of eHealth solutions. As a necessary skill in the field of eHealth, eHealth literacy is the foundation to maximize eHealth effectiveness [2]. The improvement of eHealth literacy is beneficial to the reasonable use of eHealth information resources to promote good health behaviors of the public, which is of great practical significance for promoting the health self-management ability of individuals in the information age.

To study the current situation of individuals’ eHealth literacy levels and design a corresponding intervention to improve their eHealth literacy, a systematically conducted eHealth literacy assessment is the foundation and prerequisite. Meanwhile, appropriate eHealth literacy assessment instruments can not only measure individuals’ capacity to use the eHealth tools and their benefit from these tools [10] but also recognize the special population disabled to effectively use eHealth services and experiencing “the digital divide” [11]. Up to now, the research on eHealth literacy assessment has yielded a series of research outcomes [12-15], whereas a discrepancy exists in the aspects of use scenario, applicable population, assessment themes, and dimensions of the relevant assessment models and tools. Therefore, it is necessary to sort out and analyze systematically eHealth literacy assessment instruments.

The aim of this scoping review is to comprehensively review and evaluate the characteristics, effectiveness, and limitations of existing eHealth literacy assessment instruments, providing reference and suggestions for researchers and clinicians in selecting or developing assessment instruments in the future.

This scoping review was conducted in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement [11]. A completed PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist is included in Checklist 1. Based on the research theme and considering the number of literature collections, update speed, and availability of papers, this study ultimately selected 3 databases (Web of Science, Scopus, and PubMed) and EBSCO; with regard to search strategy, the search string was (“ehealth literacy” OR “E-health literacy” OR “electronic health literacy”) AND (Assessment* OR Measure* OR Tool* OR Test* OR Instrument* OR Questionnaire* OR Psychometric* OR Screen* OR Survey*) and selected “article” as the literature category, and finally, 2972 records were obtained (searched as of June 1, 2024). The search strategies and key terms for all databases can be found in Multimedia Appendix 1.

We included studies involving the development of eHealth literacy assessment instruments, which must be published in peer-reviewed journals and written in English. If a study revolved around the theme of eHealth literacy assessment instruments, but the main purpose is not to develop a new eHealth literacy assessment instrument, then the paper was not included. In addition, if a study developed a new assessment instrument but only targeted a specific population, such as older people, the paper was also excluded. Review, commentary, or opinion papers were excluded.

All records were exported to EndNote (version X9.1; Clarivate Analytics), and duplicate records were identified and deleted. Two reviewers (CW and XC) independently evaluated the literature search results and corresponding full texts based on inclusion and exclusion criteria. If there is a difference between their selection, a discussion and consultation with the third reviewer (LC) was carried out to reach a consensus. Risk of bias was not formally assessed using a specific tool; however, the inclusion criteria were designed to minimize bias by including only original studies with clearly reported methods and outcomes.

The review authors (CW and LC) independently extracted data on development time and country, number of dimensions, number of items, scale rating level, reliability and validity testing, dimensions of assessment, and number of corresponding items. Differences in data extraction were resolved through consensus or reference to another author.

We synthesized the collated data by using descriptive statistics. We used Microsoft Excel to analyze the data.

The database search initially identified 2972 records (979, 1064, 655, and 274 in Web of Science, Scopus, PubMed, and EBSCO, respectively), and 1510 (50.8%) records were screened after removing duplicates (Figure 1). After screening titles and abstracts, 1476 (49.7%) publications were excluded. Of the total of 34 (1.1%) full-text papers screened, 13 (0.4%) papers were included in this review.

The collected 13 eHealth literacy assessment instruments’ basic information is shown in Table 1, which were developed from 2006 to 2022. The United States accounts for the most in the number of instruments with 4, followed by Denmark and China, both with 2. Based on the development of eHealth literacy concept, related technological environment, and changes in the quantity and quality of assessment tools, we divided the development of eHealth literacy assessment instruments into 2 distinct stages: the primary stage (2006) and the growth stage (2014‐2022).

In the primary stage, eHealth literacy, as a new concept, was in the early stages of comprehension and investigation, and the technological environment such as the internet was not yet mature, so the development of tools at this stage was relatively slow. Canadian and American researchers took the lead in developing eHealth literacy assessment instruments, among which eHealth Literacy Scale (eHEALS) was the first self-assessment instrument that measured eHealth literacy [2], while Research Readiness Self-Assessment (RRSA) [16] put emphasis on appraising eHealth literacy through actual operation in order to avoid the discrepancy between self-assessment and practical level.

With the rapid development of internet technology and the widespread popularity of mobile devices, as well as the further study of scholars in the field of eHealth literacy, the concept of eHealth literacy is continuously expanding, and the development of assessment tools stepped into the growth stage. At this stage, 11 scales were invented by many countries, with the United States in a leading role. Part of these scales was based on existing tools, while others are novel assessment tools on the basis of the concept and framework of eHealth literacy proposed by scholars. Stemming from eHEALS [2], 7 eHealth literacy assessment instruments were derived, including Patient Readiness to Engage in Health Internet Technology (PRE-HIT) [17], Electronic Health Literacy Scale (e-HLS) [18], Extended eHealth Literacy Scale (eHEALS-E) [19], Digital Health Literacy Instrument (DHLI) [20], eHealth Literacy Assessment (eHLA) [21], digital health literacy assessment (DHLA) [24], and eHealth Literacy Scale-Web3.0 (eHLS-Web3.0) [25], and 4 other instruments were constructed depending on newly proposed eHealth literacy framework or model, including eHealth Literacy Questionnaire (eHLQ) [22], Transactional eHealth Literacy Instrument (TeHLI) [23], Digital Health Technology Literacy Assessment Questionnaire (DHTL-AQ) [26], and digital health literacy competencies (DHLC) [27].

To clarify the assessment range of the instruments, this review referred to the extraction methods of related assessment instrument themes [50] and conducted content analysis of the included 13 assessment instruments. The measured content of the 13 instruments was categorized into 21 themes: browsing, understanding, communication, search, acquisition, application, appraisal, writing, health awareness, familiarity with health, social support, attitude, initiative, self-efficacy, confidence, sharing, familiarity with technology, health management, netiquette, privacy security, and originality protection.

These themes can be further divided into 4 areas: skill factors, psychology factors, health factors, and information factors. The skills factors contain the 8 themes: browsing, understanding, communication, search, acquisition, application, appraisal, and writing; psychology factors encompass 5 themes: social support, attitude, initiative, self-efficacy, confidence, and sharing; health factors include 3 themes: health awareness, familiarity with health, and health management; and information factors comprise 4 themes: familiarity with technology, netiquette, privacy security, and originality protection. The themes involved in each instrument are presented in Table 5. The statistical results of the assessment instruments by theme are shown in Table 6.

Among the 13 eHealth literacy assessment instruments, DHLC [27] and eHLS-Web3.0 [25] measure the most themes, covering 13 and 12 themes, respectively; RRSA [16] is the least with only 4 themes measured. In total, 12 instruments were designed to test search ability and 12 instruments for appraisal ability, and acquisition and application are also frequently incorporated into assessment content. Besides, browsing, understanding, communication, writing, attitude, self-efficacy, familiarity with technology, health management, and privacy security are also tested to some extent, while health awareness and sharing only appear in 2 instruments; familiarity with health, social support, initiative, netiquette, and originality protection are mentioned only in 1 instrument. In the future, the development of assessment instruments can increase the measurement of the psychology factors and information factors.

Our study identified 13 eHealth literacy assessment instruments through literature search, collection, screening, and collation and conducted a scoping review. Since Norman and Skinner [28] put forward the concept of eHealth literacy in 2006, the development of eHealth literacy assessment instruments has undergone great advances. Although the first eHealth literacy assessment instrument eHEALS [2] focused on the knowledge related to health literacy in the digital environment, with the advance of information technology, the concept of eHealth literacy is in an evolving process. The skills needed to use eHealth services are expanding, and the concept thereupon reflected not only an individual’s one-way ability to use technology. Subsequent models and assessment instruments were refined to adapt to the evolving digital environment and to broaden the scope of eHealth literacy measurement. To achieve this, they incorporated evaluations of digital skills, such as familiarity with mobile apps, as well as digital abilities between individuals and technology and between individuals and eHealth services. These additions aimed to comprehensively address the new demands brought about by technological advancements.

The reliability coefficient results of the 13 eHealth literacy assessment instruments indicated their satisfactory performance on reliability, being able to provide relatively reliable and stable evaluation results. Whereas in the aspects of validity, because eHealth literacy is a multidimensional and multilevel concept, different design purposes and focus of the instruments resulted in different selections of validity test categories. Most assessment instruments tested content validity to ensure that the assessment content reflects all aspects of eHealth literacy accurately. Several instruments also dealt with structural validity to examine the rationality and stability of the instruments’ internal structure. Discriminant validity, convergent validity, and external validity were seldom involved in the assessment instrument examination.

Different eHealth literacy assessment instruments adopt self-report assessment or comprehensive assessment combined with self-report and actual operation. In comparison with comprehensive assessment, self-report is limited to examine respondents’ perceptive ability to their own eHealth literacy but unable to test their actual perception and technology use ability to eHealth programs. Partial studies [20,21] have recognized the importance of including objective evaluation criteria in eHealth literacy measurement to survey participants’ real ability level. Comprehensive assessment merges subjective and objective assessment, forming an assessment with diverse evaluation dimensions and abundant evaluation levels, which is beneficial to revealing respondents’ authentic level more precisely. However, how to reasonably allocate the weight of subjective evaluation and objective measurement is not mentioned in current studies.

The main limitation of this review is that due to the limitations of the literature search strategy and database selection used, as well as the inclusion of only peer-reviewed journal papers published in English, the literature coverage might not be complete, which probably influenced the analysis results. Future work should focus on expanding current achievements.

Assessment of eHealth literacy is the premise of researching and improving eHealth literacy. This study used a scoping review method to identify 13 eHealth literacy assessment instruments from the literature and analyzed them from the perspectives of the development process, instrument characteristics, and assessment theme.

Concerning the development process, there were only 2 assessment instruments in the incipient stage in 2006, and 11 more assessment instruments were devised after 2014. Different assessment instruments’ development underwent continuous adjustment both in content and form so as to reflect the constantly renewing eHealth literacy concept.

As for assessment dimension and score range, the 13 eHealth literacy assessment instruments contain a number of dimensions ranging from 3 to 9. The self-assessment sections are rated from 4 to 6, with a 5-level being the most common.

In the aspects of reliability and validity test of the assessment instruments, the 13 instruments all conducted reliability tests, and the results revealed that the assessment instruments performed well in consistency, stability, and dependability. Except for eHEALS [2], the other 12 instruments all reported evaluation validity. On account of the difference among the instruments in their depiction of eHealth literacy’s connotation and focus, the selected validity test category of the instruments varies. In total, 12 instruments involved content validity, structural validity, discriminant validity, external validity, and convergent validity to appraise assessment instruments’ effectiveness.

In terms of assessment method, subjective assessment instruments are the most with a total of 9 evaluating in the form of self-report, exemplified by the classic assessment instrument eHEALS [2], and it gained the widest application. There are only 4 comprehensive assessment instruments, which are devised on the basis of subjective assessment and incorporate actual operational test based on the scene in order to reflect individual-related capacity objectively.

The theme analysis of the 13 assessment instruments indicated that assessment instruments targeting search, evaluation, acquisition, and application ability account for the most. Browsing, understanding, communication, self-efficacy, familiarity with technology, and privacy security were also evaluated to a certain extent, while the psychology and information factors, such as sharing, social support, initiative, netiquette, originality protection, and so on, were less concerned.

The most widely used subjective assessment tool is eHEALS [2], which is more suitable for user self-reported empirical research. The comprehensive evaluation tool, such as DHLI [20], has multiple dimensions and rich levels of evaluation, which can meet diverse evaluation needs. Overall, subjective and comprehensive assessment tools for eHealth literacy have their own advantages and disadvantages. Subjective assessment tools have a friendly evaluation method, but their test validity is relatively low. There is a risk of time-consuming and low recognition for comprehensive evaluation tools. Therefore, it is necessary to scientifically select and use evaluation tools based on different research objectives or research plan designs.