In-person interviews on the theme of internet-mobile media usages and health outcomes in rural China were conducted in Chaotian, Sichuan Province, for three weeks in June 2017. Variables included in this study were one part of the entire questionnaire. As of 2017, Chaotian was a poverty-stricken county with 25 towns and a very low level of urbanization. The percentage of rural residents in Chaotian is more than 90% [31]. The quota sampling method was adopted, and each town was assigned 50 quotas considering its individual characteristics such as age, sex, education background, and residential districts. In total, 1250 questionnaires were delivered, and all interviews were conducted by trained local interviewers.

Before the survey, all participants received written information about the study and signed a consent form if they volunteered to participate in the study. When they completed the entire questionnaire, participants were given a small present as compensation. The questionnaires with major illogical, inaccurate, and missing answers accounting for more than 15% of total questions were identified as invalid. In total, 727 rural responses were valid in this survey. After the interviews, researchers randomly selected 30 participants for in-depth interviews about the health-related behaviors’ adoption and their influence on participants’ daily life. Of all valid responses, 543 participants who identified themselves as internet users were finally included in the analysis of the C-eHEALS.

First, descriptive statistics, means, SDs, and percentages were calculated for the variables. Second, the C-eHEALS was evaluated according to the classical test theory approach. The reliability of the C-eHEALS was assessed using the Cronbach alpha coefficient (recommended value>.7) [33]. Subsequently, exploratory factor analysis and confirmatory factor analysis were conducted. In the exploratory factor analysis step, the Kaiser-Meyer-Olkin measure of sampling adequacy (recommended value>.6) and the Bartlett test of sphericity (should be statistically significant) [34] were used to test the factorability of C-eHEALS. Principal component analysis was then conducted to examine the latent properties of the eight observed items of C-eHEALS [35] and test whether the structure of the C-eHEALS has a unique pattern or is consistent with the Norman and Skinner one-factor structure [11]. Qualified factors via exploratory factor analysis should account for more than 40% of the total variance with eigenvalues>1 [36,37]. Moreover, a scree plot was used to determine the number of factors to be extracted. In the confirmatory factor analysis, we adopted structural equation modeling to evaluate the structure determined by exploratory factor analysis. The model’s goodness of fit was evaluated with the following: the Chi-square value to degrees of freedom ratio (χ²/df; recommended value<3) [38], comparative fit index (recommended value>.95), Tucker-Lewis index (recommended value >.95), root mean squared error of approximation (recommended value <.06), and standardized root mean squared residual (recommended value<.08) [39].

We thereafter tested other dimensions of the C-eHEALS’ psychometric properties using the item response theory approach. In this section, the graded response model [40], which is a generalization of the two-parameter logistic item response model (IRM) for ordinal data, was fit to the C-eHEALS. We chose IRM to help evaluate the C-eHEALS because IRM is more useful than classical test theory in providing information regarding item discriminability and difficulty [25]. Indeed, a recent study suggested that IRM should be used to evaluate the eHEALS properties [25]. Here, the graded response model was adopted over alternative IRMs because each item of the C-eHEALS has five ordered responses. In the graded response model, two types of parameters are generated: discrimination parameter alpha and difficulty or threshold parameter beta [40]. Although alpha indicates how strongly an item relates to a given latent trait theta, beta indicates the level of the latent variable a participant needs to endorse for the next higher response category, with a 50% probability. A larger beta suggests that a higher theta is required for participants to endorse a higher ordered response. Following Baker and Kim’s guidelines [41], alpha<.65 indicates low discriminability, .65-1.34 indicates moderate discriminability, and >1.34 indicates high discriminability. Next, item characteristic curves, which present the probability of participants at a given latent literacy level responding in a particular response category, were estimated for each item [42]. Then, item characteristic curves were transformed into item information curves to demonstrate how much information each item can provide. Thereafter, the test information function, which demonstrates the precision of the entire C-eHEALS along the latent trait continuum, was estimated by summing up all item information curves. In addition, the item characteristic curves were summed, in turn, to obtain the test characteristic curve, which represents the expected score of the C-eHEALS.

Subsequently, following the suggestion of a previous study [14], we also tested the criterion validity of the C-eHEALS via hypothesis testing. We hypothesized that the C-eHEALS score is positively associated with SOHISB among rural participants. Because SOHISB provides count data, Poisson regression was adopted to perform the estimation [43].

Details of the characteristics of rural participants are listed in Table 1. Among the 543 participants, the mean age was 40.37 (SD 9.19) years, ranging from 18 to 70 years; men accounted for 58.56% of the participants. Most participants were married (83.43%). As for the highest education level, almost half of the participants listed junior middle school (46.49%), 27.81% listed primary school and below, 16.39% participants listed senior middle school, and 9.21% listed junior college and above. In terms of employment, approximately one-third of the participants (37.57%) had farming jobs, 17.31% had nonfarming jobs within the county, 27.44% had nonfarming jobs outside the county, 7.37% were unemployed, and 10.31% reported other jobs.

Although all samples were recruited in one county via quota sampling, which did not have an ideal representativeness, we compared the internet users in Chaotian and overall China as per a recent report [44]. At the end of 2015, 55.2% Chinese rural internet users were men [44], which was relatively equivalent to the sample in this study. However, the overall Chinese rural internet users showed a higher educational background (20.8% completed primary school and below, 51.9% completed junior middle school, 21.4% completed senior middle school, and 6% completed junior college and above) than the sample in Chaotian. Additionally, regarding the employment status, the overall Chinese rural internet users had fewer people in the farming occupation (15.8%) than the Chaotian sample. The differences between the Chaotian sample and overall Chinese internet users are not surprising as Chaotian, the survey site we chose, was a typical impoverished county when the investigation was conducted.

As for the 11 online health information–seeking behaviors, the least performed behavior was scheduling appointments online (3.31%) and the most was reading or sharing health information via social media (42.91%). As shown in Table 2, the mean of the items in the C-eHEALS ranged from 3.26-3.46 of 5.

The C-eHEALS had excellent reliability (Cronbach alpha=.834). Both the Kaiser-Meyer-Olkin measure of sampling adequacy (.829; Table 3) and Bartlett test of sphericity (1556.34 (df=28), P<.001) showed a good fit to the data, allowing for exploratory factor analysis [34]. Exploratory factor analysis using principal component analysis resulted in a one-factor solution with an initial eigenvalue of 3.159, accounting for 91.8% of the variance (Table 3). The scree plot also showed a one-factor structure (Figure 1). As shown in Table 3, all items loaded above .5, varying from .576 (C-eHEALS1) to .706 (C-eHEALS3). Thus, a single factor was retained.

Confirmatory factor analysis was run to verify the one-factor structure obtained from exploratory factor analysis. Results suggested that the C-eHEALS has an excellent one-factor structure (χ²/df=1.813, comparative fit index=0.993, Tucker-Lewis index=0.985, root mean squared error of approximation=0.039, standardized root mean squared residual=0.022). Thus, the structure of the C-eHEALS is consistent with the original eHEALS proposed by Norman and Skinner [11] and the traditional Chinese version of the eHEALS [17].

All eight items of the C-eHEALS were fit to a graded response model. The item parameter estimation and item fit statistics are displayed in Table 4. The discrimination parameters (alpha) ranged from 1.32 to 2.3, indicating that all items discriminated between low and high levels of eHealth literacy well. Only item C-eHEALS6 has a moderate alpha value (1.32), while the other seven items have high discriminability.

Difficulty parameter (beta) estimates indicated that the C-eHEALS is more sensitive at the lower range of latent trait theta because all mean beta (Beta_M) values were lower than 0. The beta values of C-eHEALS1 and C-eHEALS5 were unevenly distributed across the trait range, indicating that most participants were unlikely to endorse lower response options. The other six items (C-eHEALS2, C-eHEALS3, C-eHEALS4, C-eHEALS6, C-eHEALS7, and C-eHEALS8) were distributed evenly across the trait range, suggesting that these items differentiate participants from low through high trait levels.

In addition to these results, item characteristic curves are included in Figure 2. These plots show the probability that a participant selects a particular response category at a given level of the latent construct. It was observed that the response categories were distinguishable and monotonically related to the latent trait theta for all items.

Test information function, as reported in Figure 3, reveals that theta values<–3 and >2.5 are poorly represented relative to the rest of the trait range. This is true for seven items (C-eHEALS1, C-eHEALS2, C-eHEALS4, C-eHEALS5, C-eHEALS6, C-eHEALS7, and C-eHEALS8). Only the item information curve of C-eHEALS3 represented a significant fluctuation when theta levels are approximately between 1 and 3. Figure 4 presents the test characteristic curve, which indicates that 95% of randomly selected participants are expected to score between 18.4 and 33.6.

The results of Poisson regression of the relationship between the C-eHEALS score and SOHISB are displayed in Tables 5 and 6. The odds ratio of the C-eHEALS score is significantly positive, which indicates that participants with higher C-eHEALS scores will have more diverse online health information–seeking behaviors. Hence, the C-eHEALS was shown to have good criterion validity for the Chinese rural population. In addition, coefficients of demographic variables (sex, age, and marital status) do not shown statistical significance, which means the SOHISB does not vary significantly with sex, age, and marital status. However, five dummy variables of socioeconomic status (educational background and employment status) presented significant positive associations with SOHISB. These results are consistent with the knowledge gap hypothesis [45] that implies that individuals will have more possibility to access the information channels to obtain useful information if they are living with higher socioeconomic status. More importantly, even after controlling for all these variables, the C-eHEALS score still plays a positive role in accessing internet technologies to seek health information. The findings in Tables 5 and 6 also suggest that eHealth literacy may have strong practical significance for its implementation to overcome social disparities.

This study investigated the psychometric properties of the simplified C-eHEALS in a Chinese rural population via both classical test theory and item response theory approaches.

Classical test theory analyses demonstrated that the C-eHEALS has good reliability and validity for the rural population in China. The internal consistency of the C-eHEALS was .834, which was comparable to the original eHEALS’ alpha value of .88 reported by Norman and Skinner [11]. Exploratory factor analysis results revealed that the C-eHEALS has a one-factor structure, which is consistent with the structures of the original eHEALS [11] and its traditional Chinese version [17]. Surprisingly, the factor accounted for more than 91.8% of the variance, which is much higher than that reported in previous findings [11,14,16,17,25]. Furthermore, referring to previous studies’ suggestions [25,29], this one-factor structure also fit well in the confirmatory factor analysis.

Results of the item response theory revealed that response options could differentiate between latent trait levels of all eight C-eHEALS items. The entire instrument provides less information only at extremely low levels (theta<–3) and high levels (theta>2.5) of the latent trait. These results indicate that C-eHEALS is an excellent measure for capturing participants’ ability. Two items (C-eHEALS1 and C-eHEALS5) are more sensitive at the lower range of the latent trait, and the other six items represent excellent discriminability for participants from low through high trait levels.

This study also demonstrated that the C-eHEALS has good criterion validity. A previous study indicated that individuals with a higher level of health literacy will report a larger scope of health information sources [46]. Hence, the diversity of information access channels should be considered as one criterion of better literacy. We also hypothesized that the eHealth literacy score is positively associated with the SOHISB among rural populations. Indeed, controlling for confounding variables, Poisson regression results supported the hypothesis and revealed that rural people’s information-seeking behavior could be cultivated with adequate eHealth literacy.

The eHEALS is a validated instrument in diverse language environments [14,16,17,25,27]. In mainland China, it was first introduced in 2013 [47] and has received attention in recent years [48,49]. However, those studies were limited to reporting sophisticated psychometric properties of the eHEALS [47-49]. In addition, rural populations were ignored in previous research. Given the currently targeted poverty alleviation strategies, the campaign launched by the Chinese central government to enhance residents’ health literacy status over the next decade [8], and the obvious internet access gap between rural and urban areas [19], both community- and county-level health promotion campaigns should emphasize on health education for rural populations to mitigate large rural-urban disparities.

Health information diffused via the internet should be appropriately evaluated by individual internet users, which can be strengthened by health literacy. For future health literacy-related studies concerning Chinese rural populations, this study provides a useful instrument that can be adopted in survey studies. Moreover, highlighting aspects of health literacy specific to the internet context (eg, practical skills) over other aspects such as the perception description in the eHEALS should be considered for future research.

Findings about the criterion validity also revealed that eHealth literacy is a key element to promote rural residents’ access to ICTs for health-related information. Enhancing eHealth literacy might help rural residents overcome their inadequate resource acquisition capacities restricted by local economic recession. It is worth noting that eHealth literacy may also lead to a new digital divide between rural and urban populations. Besides the information-seeking behavior, some other online health-related practices, like health management [4] and mobile app-assisted self-care [50], may present varied implementation practices among people with different socioeconomic statuses. Future studies should design comparisons between populations living in areas with different levels of urbanization.

There are two main limitations in this study. First, all participants were recruited from one poverty-stricken county in China via quota sampling, which cannot well represent the diverse situations of China’s rural-urban disparities and the entire Chinese rural population. The psychometric properties of the C-eHEALS may vary under different economic development statuses. Hence, future studies aiming to replicate our findings in other samples are highly encouraged. Second, this study had a cross-sectional design, and hence, we were unable to calculate test-retest reliability or predictive validity estimates [51]. Future studies may address this limitation via longitudinal designs.

The C-eHEALS was found to have a robust one-factor structure with excellent discriminability among the Chinese rural population. This scale is helpful for health education practitioners and health professionals to properly measure and understand rural people’s eHealth literacy before launching health campaigns. We hope to encourage health researchers who conduct studies in eHealth to carefully investigate policy effects on rural people.