An initial evaluation index system was developed through a literature review and focus group discussions (FGDs). This was followed by a 2-round Delphi expert consultation involving 15 experts. The FGDs were convened by a 13-member research team led by the principal investigator, comprising 2 senior researchers, 4 midlevel researchers, 2 junior researchers, and 5 graduate researchers (2 doctoral and 3 master’s candidates). These discussions served 2 specific purposes: first, during the initial stage, they were used to screen and refine the item pool generated from the literature review, where participants discussed the relevance and clarity of potential items; second, during the Delphi process, they were used to facilitate discussion and decision-making regarding the adoption of expert opinions, helping to interpret disagreements and reach consensus on item modifications. Thus, the FGD data were iteratively used to inform item refinement and integrate expert feedback, rather than being analyzed through formal qualitative methods such as thematic or content analysis. By integrating qualitative analysis with quantitative computation, the initial framework comprising “6 primary indicators, 13 secondary indicators, and 28 tertiary indicators” was progressively refined into a scientifically structured system featuring “6-13-27” indicators. Finally, the reliability and validity of the questionnaire were verified through a cross-sectional survey. The specific procedural design is illustrated in Figure 1.

A comprehensive literature search was performed in the PubMed, Web of Science, Cochrane Library, CNKI, and WanFang databases to collect initial evaluation indicators using the following keywords:

“older adults” OR “older people” OR “elderly” AND “digital health literacy” OR “electronic health literacy” OR “eHealth literacy” OR “digital health” OR “information literacy” AND “indicator” OR “index” OR “scale.”

The search covered literature published from its inception to February 28, 2025. The results revealed that previous studies focused primarily on assessing the ability to access, evaluate, and apply health information, and ignored the influence of self-efficiency, as well as social and environmental elements, on DHL among older adults. Based on these findings, we developed a new comprehensive questionnaire with particular emphasis on the above dimensions, considering both the individual capacity and the environmental support of the older population.

This study used the Delphi method to administer expert consultation questionnaires via an online platform [37] to gather perspectives among participants, thereby enhancing the objectivity and scientific validity of the outcomes. While the Delphi methodology imposes no fixed threshold for expert panel size, increased participant numbers strengthen result stability [38]. A thorough review of the literature indicates that a panel of 4 to 16 experts generally yields sufficiently reliable findings for Delphi consultations. When the number of selected experts approaches 15, further increases contribute minimally to prediction precision. Consequently, this study ultimately enrolled 15 experts as participants for the consultation.

eHealth literacy represents a complex, multidisciplinary issue. The experts selected for consultation should possess authoritativeness, representation, and reliability, particularly to demonstrate substantial knowledge or experience relevant to the research topic. Based on the study’s objectives and content, the Delphi expert panel was recruited through purposive sampling, leveraging the research team’s established professional networks in geriatric care, clinical practice, and academic research. Invitations were extended to individuals who met the following predefined eligibility criteria: (1) work experience of 10 years or more in fields including geriatric medicine, geriatric nursing, older adult care services, or community management; (2) a bachelor’s degree or higher; (3) an intermediate-level technical title or higher; and (4) voluntary participation in the study with the ability to complete and return questionnaires promptly. Ultimately, a multidisciplinary panel of 15 experts including clinicians, nurses, health care administrators, and researchers, agreed to participate and completed both rounds of the Delphi consultation.

Based on the findings from the literature review and qualitative interviews, the initial evaluation index system for assessing older adults’ EHL was developed into a consultation questionnaire. This instrument comprises four main sections: (1) an introduction to the research, including the purpose of the study, instructions for completing the questionnaire, details on how and when to submit it, contact information, and acknowledgments; (2) demographic information of experts; (3) the core section (expert consultation questionnaire on DHL evaluation indicators for urban community-dwelling older adults); and (4) an expert authority questionnaire, which includes expert self-assessments of their authority and familiarity with the questionnaire content according to theoretical analysis, work experience, peer understanding, and intuition.

Within the questionnaire’s primary section, the experts rated the importance of each indicator across all hierarchical levels via a 5-point Likert scale (ranging from 1=“Not Important” to 5=“Extremely Important”). Additionally, experts were invited to provide specific suggestions for adding, deleting, or modifying indicators in designated comment fields.

Two rounds of consultation were conducted via email and WeChat. A mean importance score of more than 3.50, a coefficient of variation (CV) of 0.30 or less, and a total score rate of more than 20% were the inclusion criteria [39,40]. Entries were adjusted on the basis of discussions within the research team, incorporating expert modifications. After 2 rounds, the final DHL questionnaire for older adults included 6 dimensions and 27 items (Table 1). The detailed results of scoring of experts were presented in the Multimedia Appendix 1.

Participants and Sample Selection Permanent residents aged 60 years and older in a city in Sichuan Province (defined as people living in the city for 6 months or more, without distinguishing local household registration); basic understanding ability, ability to understand and fill in or answer the questions in the questionnaire; and voluntary participation in this study.

The study was approved by the ethics committee on Biomedical Research, West China Hospital of Sichuan University (1044). All participants were informed of the study’s purpose, and procedures, and provided voluntary consent to participate. Delphi experts received an honorarium for their participation, while cross-sectional survey participants took part voluntarily and received no compensation. The survey adhered to the principles of anonymity and confidentiality, and all responses were collected anonymously.

The Cronbach α coefficient and Spearman-Brown split-half reliability are the most commonly used methods for testing the internal consistency reliability of questionnaires [43]. The split-half reliability was assessed by dividing the 27 items into 2 halves using the odd-even method. Generally, a Cronbach α coefficient greater than 0.500 for each dimension, along with a total Cronbach α coefficient and Spearman-Brown split-half reliability greater than 0.7 indicate acceptable reliability.

For validity testing, since there is currently no universally established “gold standard” instrument specifically designed to measure comprehensive DHL within the contemporary Web 3.0 paradigm for older adults, the eHEALS was used as the criterion tool in this study. Criterion validity was assessed by examining the correlation between the total score of our scale and the eHEALS score. In addition, known-groups validity was evaluated. Participants were divided into 2 groups based on their educational attainment (high school or above vs junior high school or below), as educational level has been consistently identified as a key determinant of DHL in previous literature [44]. An independent-samples 2-tailed t test was conducted to compare the mean scale scores between the 2 groups. Construct validity, assessed through factor analysis, was not performed in this phase of the study, primarily due to the limited sample size. This limitation is acknowledged in the “Discussion” section and will be addressed in future research with a larger sample.

Data management and statistical analyses were conducted using Microsoft Excel and the SPSSAU online platform (Beijing Qingsi Technology Co, Ltd). Expert demographic characteristics were described with frequencies (percentages) for categorical variables and means (SDs) for continuous variables. Expert engagement was evaluated through questionnaire response rates and the proportion of experts providing substantive feedback. Expert authority was quantified via the authority coefficient (Cr=[Ca+Cs]/2). The judgment basis coefficient (Ca) was derived from the expert’s self-assessment of the influence level (high, medium, or low) across 4 dimensions, with scores assigned as shown in Table 2.

The familiarity coefficient (Cs) was assigned based on the expert’s self-rated familiarity with the subject, using a 5-point scale (1.0=“very familiar,” 0.75=“familiar,” 0.5=“generally familiar,” 0.25=“somewhat unfamiliar,” and 0=“very unfamiliar”). Both components were collected via a self-assessment questionnaire completed by the experts. Consensus was assessed via the CV and the Kendall concordance coefficient (w).

The Delphi consultation involved 15 experts from 13 universities and tertiary-level hospitals located in Sichuan Province, Beijing, Shanghai, Hunan Province, Shandong Province, and Xi’an Province, China. All 15 (100%) completed both rounds of the Delphi consultation. Among the 15 experts, 6 (40%) were male participants and 9 (60%) were female participants. The participants were aged 36 to 56 years (mean 45.93, SD 6.17), with professional experience ranging from 10 to 34 years (mean 21.07, SD 8.21). Twelve (80%) experts held a master’s degree or higher, and 13 (86.67%) possessed associate senior professional titles or above. Regarding administrative roles, 1 (6.67%) served as a hospital president, 8 (53.33%) held positions as directors or deputy directors of clinical departments, and 1 (6.67%) was the head nurse of a ward unit. The characteristics are presented in Table 3.

A questionnaire return rate exceeding 70% is generally considered indicative of good retrieval and high expert engagement [45]. In the first round of this study’s expert consultation, 15 expert correspondence questionnaires were distributed, and all 15 were returned, yielding a return rate of 100%. Among these, 7 (46.67%) experts provided comments. In the second round, 15 questionnaires were distributed, all 15 were returned (100% return rate), and 4 (26.67%) experts provided comments. These results demonstrate a high level of expert engagement in the consultation process.

The expert authority coefficient (Cr) is determined jointly by Cs and Ca. The mean Cr values for the 2 consultation rounds were 0.748 (SD 0.096) and 0.768 (SD 0.089), both exceeding the acceptable threshold of 0.7 [45]. Therefore, the expert authority level for this consultation is deemed acceptable.

Following 2 rounds of consultation, expert opinions converged, demonstrating high acceptance of all indicators. The CV in the first round ranged from 0.141 to 0.207, whereas in the second round it ranged from 0.141 to 0.225. The Kendall W coefficient tests for both consultation rounds yielded statistically significant results (P<.001 and P=.005, respectively), indicating strong consensus among expert evaluations.

In the first Delphi round, the definition of 1 primary indicator was revised, and the conceptual definition of “confidence and trust in application” was redefined as “trust in self-efficacy and online health information.” One secondary indicator (“belief in online health” was updated to “belief in online health information”) underwent revisions, while 1 tertiary item was eliminated because of content overlap with the 12 tertiary items that were modified.

In the second round, 1 primary indicator definition was refined, and the definition of the primary dimension “external support” was refined from “evaluation of external factors” to “perception of external factors.” One secondary indicator was amended, and minor wording optimizations were implemented for 6 tertiary items.

All indicators achieved a mean importance score exceeding 3.5, with a full-score rate surpassing 20%, meeting predetermined criteria except for a few items demonstrating CV values below 0.30. Given the critical significance of privacy exposure and digital technology infrastructure in the digital era for DHL, the item (“5.1.3 I regularly clear browsing history”) was retained following rigorous FGDs. A strong perspective held that, in the context of managing online health information, this behavior concretely reflects privacy protection awareness—a critical facet of the “safety” dimension in DHL. Its high importance score and theoretical alignment supported its inclusion.

Through research team deliberation guided by predefined inclusion criteria and expert feedback, the final hierarchical framework for assessing older adults’ DHL comprised 6 primary indicators, 13 secondary indicators, and 27 third-level indicators (Table 1).

The pretest yielded 332 valid questionnaires. The participants’ ages ranged from 60 to 82 years, with a mean age of 65.4 (SD 5.2) years. The cohort comprised 175 (52.7%) males and 157 (47.3%) females. Married individuals constituted the majority (n=276, 83.1%). The education level was predominantly junior high school or below (n=243, 73.2%), and farming was the primary occupation (n=165, 49.7%). Most households consisted of 3 to 5 members (n=233, 70.1%), and coresiding with children was the most common living arrangement (n=176, 53%). The detailed characteristics are presented in Table 4.

The instrument is based on a 5-point Likert scale. The total score is derived from the sum of all 27 items, with a theoretical range of 27 to 135 points. In the validation phase, an equally weighted summed score was used for clarity and ease of interpretation, with each item contributing equally to the total.

In the cross-sectional survey, the mean total DHL score among older adults was 70.13 (SD 23.53), with a median score of 67.0 (IQR 51.0‐91.0). The mean item score was 2.60 (SD 0.87), with a median item score of 2.48 (IQR 1.89‐3.37), based on a 5-point Likert scale.

This study primarily assessed the internal consistency reliability and split-half reliability of our DHL for older adults. The IBM SPSS software yielded the following results: internal consistency reliability: The overall Cronbach α coefficient for DHL was 0.984, exceeding the acceptable threshold of 0.70. All dimensions demonstrated satisfactory internal consistency, with Cronbach α coefficients exceeding the threshold of 0.70. The detailed results are presented in Table 5.

The split-half reliability was assessed by dividing the 27 items into 2 halves using the odd-even method. The Cronbach α coefficients for these 2 subsets were 0.968 and 0.967, respectively. The correlation coefficient was 0.952. The Spearman-Brown split-half reliability coefficient for the full scale was 0.922. These results collectively demonstrate excellent internal consistency reliability for DHL.

Criterion validity was used to evaluate the validity of the DHL. The Chinese version of the eHEALS serves as the reference criterion, as it represents one of the most widely used and validated instruments in this domain [26]. Its established reliability in older Chinese populations (Cronbach α=0.988) [46] supports its suitability as a benchmark. Analysis revealed a strong Pearson correlation coefficient of 0.891 (P<.001) between the DHL scale and the eHEALS, demonstrating good concurrent validity for the newly developed scale.

To further examine the construct validity of the scale, a known-groups validity analysis was conducted. Consistent with existing literature [44], educational attainment is a well-established predictor of DHL. Participants were therefore divided into 2 groups based on their level of education: “high school or above” education (n=89) and a group with “junior high school or below” education (n=243). The results of an independent samples t test revealed a statistically significant difference between the groups (t₃₃₀=−4.37; P<.001). This finding indicates that the newly developed scale can effectively discriminate between groups with theoretically different levels of DHL, thereby providing strong evidence for its construct validity.

In this study, an assessment framework and corresponding scale for evaluating the DHL of older adults were developed and underwent preliminary validation. The framework comprises 2 dimensions—individual competency and external support—with 6 first-level indicators, 13 second-level indicators, and 27 third-level indicators, covering the core dimensions of DHL among older adults. The total score ranges from 27 to 135, derived from the sum of all 27 items rated on a 5-point Likert scale. To offer an initial reference for application, an interpretive framework was developed based on the percentile distribution of the pilot sample (N=332). Specifically, total scores below the 25th percentile were preliminarily categorized as “needing support,” scores between the 25th and 75th percentiles as “having foundational competency,” and scores above the 75th percentile as “proficient.” It must be emphasized that this framework and its cutoff values are derived from the pilot data and serve as provisional, practical benchmarks. Their generalizability and optimal thresholds require further validation in future studies with larger, more representative populations.

The study fully considers the interplay between socioecological levels, and the finalized “individual competency-environmental support” dual-pathway index system transcends the limitations of conventional measurement tools that focus solely on individual skills. At the individual competency level, the newly incorporated “confidence and trust in application” dimension emphasizes self-efficacy and trust in online health information, capturing psychological and cognitive influences on digital health behaviors. The “critical evaluation skills,” “application skills,” and “security and privacy” dimensions delineate competencies in information filtering, authenticity verification, and privacy protection, aligning with digital health demands in the Web 3.0 era. At social and community levels, the influence of family, friends, and peers on an individual’s health-related behavior has been recognized [47,48]. In the DHL scale, the wider cultural, social, and environmental level elements are included. The external support dimension encompasses multiple stakeholders (community, government, institutions, family, and so on), where indicators such as “health care services,” “community-based technology training,” and “broadband internet” specifically address practical barriers to digital inclusion, providing a clear direction for policy development and service design.

There are many measurements in the field of DHL. However, only a few studies have explored the influencing factors of older adults’ DHL from the perspective of social capital [49]. Compared with existing instruments, the key distinction of this framework lies in its systematic incorporation of “environmental support” as an independent dimension, moving beyond a focus solely on individual skills. While widely used tools such as eHEALS emphasize personal operational and cognitive abilities, this study integrates external factors, including community, family, and social support, aligning more closely with health ecological theory and digital inclusion perspectives. Methodologically, the use of multiround Delphi consensus combined with qualitative data from focus groups enhanced the content validity and cultural relevance of the indicators, addressing limitations associated with purely literature-driven or statistically screened item pools, which are commonly used in the research of tool construction [25,50].

This assessment framework has dual practical significance. First, it establishes a standardized screening tool for assessing older adults’ DHL. The external support was first included in the DHL evaluation index system. Second, it promotes multistakeholder collaboration to enhance digital health capabilities. In addition to increasing eHealth skills, it is important to consider the role of trust, self-efficacy, motivation, social influence, and support in the use of digital health tools [51]. In terms of government, policies on national digital infrastructure (eg, free Wi-Fi for all), health service budgets, and resources for investing in and supporting digital health rollout can be used to mitigate digital inequalities [11], and communities and families can implement targeted training and companion services [52]. This integrated approach empowers older adults to transition from “digitally vulnerable groups” to “active health self-managers,” advancing the achievement of healthy aging goals.

Despite yielding noteworthy findings, this study has several limitations. First, methodologically, the scale relies on self-report and lacks item weighting, which may affect the objectivity of the results and the discriminant validity among core competencies. Future work will combine behavioral observation with confirmatory factor analysis to cross-validate the measurement approach and explore weighting schemes. Second, the single-center, cross-sectional design limits the generalizability of the findings and prevents examination of the tool’s ability to predict health behaviors. A multicenter longitudinal study is planned to expand sample diversity and track the relationship between scale scores and actual health behaviors. Third, the current framework lacks the capacity for dynamic assessment. Given the rapid advancement of digital technologies and the evolving health needs of older adults, the existing indicators may not remain timely or fully applicable in the long term [53]. A standing revision panel comprising experts in geriatrics, digital technology, and public health will be established to implement periodic evaluation and content updates, ensuring the tool’s continued relevance amid technological change.

The dual-dimensional “individual competency–environmental support” framework developed in this study offers a structured tool for systematically identifying variations in digital health competencies among older adults. It highlights the combined role of individual and environmental factors in shaping DHL and provides a standardized approach to assess key competencies and contextual barriers. The framework can be used for needs identification, community-based intervention planning, and evaluation of digital inclusion initiatives, while also supporting the design of targeted, multistakeholder interventions. Importantly, this study aligns closely with national strategies such as “Healthy China” and “Active Aging,” offering evidence-based support for policy implementation from the perspective of digital empowerment and inclusive development. By promoting DHL among older adults, the research contributes to enhancing their ability to manage health autonomously and participate socially, thereby supporting the policy goal of building an aging-friendly digital health environment and an inclusive aging society. Future studies should validate the framework across diverse populations and settings to strengthen its applicability and policy relevance, ensuring its continued use in supporting and refining national healthy aging strategies.