This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Primary health care workers (PHCWs) are a major force in delivering basic public health services (BPHS) in rural China. It is necessary to take effective training approaches to improve PHCWs’ competency on BPHS. Both electronic learning (e-learning) and blended learning have been widely used in the health workers’ education. However, there is limited evidence on the effects of blended learning in comparison with pure e-learning.
The aim of this study was to evaluate the effects of a blended-learning approach for rural PHCWs in improving their knowledge about BPHS as well as training satisfaction in comparison with a pure e-learning approach.
The study was conducted among PHCWs in 6 rural counties of Hubei Province, China, between August 2013 and April 2014. Three counties were randomly allocated blended-learning courses (29 township centers or 612 PHCWs—the experimental group), and three counties were allocated pure e-learning courses (31 township centers or 625 PHCWs—the control group). Three course modules were administered for 5 weeks, with assessments at baseline and postcourse. Primary outcomes were score changes in courses’ knowledge. Secondary outcome was participant satisfaction (5-point Likert scale anchored between 1 [strongly agree] and 5 [strongly disagree]).
The experimental group had higher mean scores than the control group in knowledge achievement in three course modules: (1) module 1: 93.21 (95% CI 92.49-93.93) in experimental group versus 88.29 (95% CI 87.19-89.40) in the control group; adjusted difference, 4.92 (95% CI 2.61-7.24;
Among PHCWs in rural China, a blended-learning approach to BPHS training could result in a higher knowledge achievement and satisfaction level compared with a pure e-learning approach. The findings of the study will contribute knowledge to improve the competency of PHCWs in similar settings.
In rural China, health services were delivered by a 3-tiered system consisting of county-level health care facilities, township hospitals, and village health clinics [
Human resources is the crucial core of a health system, especially with regard to quantity and quality [
The increased popularity of the Internet and the growth of computer processing power during the past decade have provided opportunities for innovation and new approaches for training [
On the basis of the fact that most PHCWs in rural China need more effective training modes to improve their knowledge on BPHS, our study aimed to evaluate the effects of a blended-learning approach in improving BPHS knowledge among PHCWs in comparison with a pure e-learning approach.
A comparative study was conducted in 3 cities (Yichang, Ezhou, and Xianning) in Hubei Province between August 2013 and April 2014. A multistage clustering sampling method was used to select participants in this study. In the first stage, according to their gross domestic product (GDP) rank in 2013 in Hubei Province, the cities of Yichang, Xianning, and Ezhou city were selected (low: Ezhou; medium: Xianning; high: Yichang). In the second stage, 2 counties with similar background characteristics in each city were selected; a total of 6 counties (Yiling and Zhijiang from Yichang city, Xianan and Chibi from Xianning city, and Huarong and Liangzihu District from Ezhou city) with 60 township centers were approached. In the third stage, the 2 counties in each city were randomly allocated to 2 groups, and therefore 3 counties, including 29 township centers were included in the blended-learning group (Zhijiang, Xianan, and Huarong counties; experimental intervention, 612 participants), and the other 3 counties, including 31 township centers, were in the pure e-learning group (Yiling, Chibi, and Liangzihu counties; control intervention, 625 participants). The selected counties in each city were at an average distance of more than 43 km.
Included participants were PHCWs, either from township centers or village clinics within the administrative prefecture of each selected township, who are currently providing BPHS to rural residents. Exclusion criteria were refusal to provide informed consent, lack of space to attend the training, lack the basic computer skills, or lack of an Internet connection.
Three course modules were developed based on the BPHS contents [
All participants were enrolled in the study for an overall period of 5 weeks (1 week for trainees to familiarize themselves with training platform; 3 weeks for the theoretical learning; and 1 week for the case study). Before theoretical learning, all trainees could have access to the manual about training platform for 1 week and receive training or guidance for using the platform. I For the sake of consistency between the two groups, all study subjects were required to complete the theoretical learning of the three course modules first before starting the case studies. During the intervention implementation period, there was no regular meeting held at the county CDC (Center for Disease Control and Prevention) to reduce contaminations between the two intervention groups. Two facilitators were present during the training sessions of both groups for assistance and to answer questions. The details for the interventions are as follows.
The pure e-learning group received Internet training on the training platform. Theoretical learning was presented in the format of Microsoft PowerPoint with 5-6 questions inserted into the slides, and a synchronous audio explanation was attached in each slide. Case studies consisted of 3 video sessions in which “real-world” examples or cases were delivered by a lecturer. Each case-study video, consisting of 4-5 cases, was about 30 min in length. All learning activities had to be completed independently at a self-paced rate. Two discussion forums were developed on the training platform, for the theoretical learning and case studies respectively. The discussion forum for the theoretical learning was set to separate groups, meaning only the same group learners could discuss and talk to each other, to reduce contaminations between the two intervention groups [
Participants in the blended group studied the same PowerPoint-based theoretical materials (available at the same training platform) during the same period. After that, participants received the handouts of all case-study materials for self-studying 4-5 days and attended 1-day (8-h) face-to-face case-study training. All cases were administered on the day by the same lecturers as in the videos in the meeting room at county CDC. PHCWs were encouraged to discuss the cases with educators and other physicians during the face-to-face training.
Assessment instruments consisted of two parts: the same pre- and posttest multiple-choice questions (MCQ) test in a different order to evaluate knowledge achievement, and a questionnaire to evaluate trainees’ satisfaction. Each trainee at the start answered the pretest questionnaire to gain access to the three training course modules for 4 weeks. After 4 weeks of learning, trainees were asked to complete the posttest MCQ for three course modules. Due to the various dropouts from each course module, there were different numbers of participants in each course training. After the completion of the three course modules, all participants were asked to fill out an online evaluation questionnaire during the following week.
A similar pre- and a posttest questionnaire was developed to measure trainees' knowledge achievement in each course module. A total of 3 knowledge MCQ tests were developed, consisting of a 10-item MCQ test in course module 1, a 15-item MCQ test in course module 2, and a 20-item MCQ test in course module 3. Both groups finished the precourse MCQ tests online within 60 min (each MCQ test under 20 min). Experimental group learners finished the post-course MCQ tests onsite, and control group learners finished them online—both within 60 minutes. All questions were scored as one point per correct response and zero points for an incorrect response. Scores were changed as a percentage of questions answered correctly.
An additional 8-item questionnaire was administered to all participants to evaluate their experience with the courses and training methods on a 5-point Likert scale from 1 (strongly agree) to 5 (strongly disagree) after finishing the three course modules (both the theoretical learning and case studies). The questionnaire was piloted with 52 PHCWs and revised accordingly to ensure that the questions could be understood and answered well by all respondents. Cronbach alpha for the questionnaire was .975 according to the pilot study. Subjects who participated in the pilot test were excluded from the final analysis.
The primary outcome was the difference between the control and experimental intervention group in knowledge achievement (measured by baseline and postcourse MCQ tests). The secondary outcome was the difference in trainees’ satisfaction with the courses and training methods between the control and experimental intervention groups (measured by an 8-item evaluation questionnaire)..
The information regarding baseline knowledge, possible gains, and intracluster (intraclass) correlation coefficient was obtained from our pilot study work to calculate the sample size and power calculation. A total of 56 clusters (township centers) are needed to detect a knowledge gain of 5% in the experimental intervention compared with the control intervention using a 2-sided test, an alpha level of 5%, 80% power, assuming a standard deviation of 20, an intracluster (intraclass) correlation coefficient of .06, and expecting a mean cluster size of around 20.
Data was presented as mean with 95% CI. Responses to the baseline and postcourse assessments were scored, and comparisons between the 2 groups were made. The MCQ postscores were compared between the two groups using a multilevel linear mixed model, with intervention group, time of assessment (baseline or postcourse), and intervention × time interaction as fixed effects and township centers and participants as random effects. For evaluating participants’ satisfaction with the training modalities, the responses were computed on a 5-point Likert scale from 1 (strongly agree) to 5 (strongly disagree). Because very few participants chose scores of 3, 4, or 5, in the analysis, we combined responses with scores of 3, 4, and 5 into a single category “neutral or disagree.” Univariate logistic regression analysis was used to calculate the odds ratios (ORs) and 95% CI for comparing the difference between the two groups on each item of the questionnaire.
All comparisons were 2-sided and were considered statistically significant at
This study was approved by the Ethics Committee of Tongji Medical College, Huazhong University of Science and Technology. Written informed content was obtained from all study subjects before the study.
A total of 1237 PHCWs were recruited (
Demographic characteristics of the participants.
Variable | Blended-learning group (N=569) | Pure e-learning group (N=563) | DF | χ2/ |
|||
Mean age (SDa), years | 41.67 (11.08) | 41.98 (9.58) | 1130 | 0.5 | .59 | ||
≤29 years | 72 (12.7) | 55 (9.8) | 3 | 7.4 | .06 | ||
30-39 years | 184 (32.3) | 182 (32.3) | |||||
40-49 years | 165 (29.0) | 200 (35.5) | |||||
≥50 years | 148 (26.0) | 126 (22.4) | |||||
Male | 291 (51.1) | 320 (56.8) | 1 | 3.7 | .06 | ||
Female | 278 (48.9) | 243 (43.2) | |||||
Technical secondary school or belowb | 453 (79.6) | 435 (77.3) | 2 | 1.1 | .57 | ||
Junior college | 101 (17.8) | 109 (19.4) | |||||
Undergraduate or above | 15 (2.6) | 19 (3.4) | |||||
Western medicine | 308 (54.9) | 346 (61.5) | 4 | 6.2 | .18 | ||
Nursing | 129 (23.0) | 113 (20.1) | |||||
Preventive medicine | 52 (9.3) | 40 (7.1) | |||||
Traditional Chinese medicine | 29 (5.2) | 31 (5.5) | |||||
Other | 43 (7.7) | 33 (5.9) | |||||
aSD: standard deviation.
bTechnical secondary school or below: illiterate or primary school, middle school, high school, or technical secondary school.
c χ2/
Study flow diagram.
Baseline knowledge scores of the three course modules between experimental and control group were similar. After the interventions, there were more gains in the experimental group than in the control group: (1) Course module 1: postcourse mean, 93.21 (95% CI 92.49-93.93) in the experimental group versus 88.29 (95% CI 87.19-89.40) in the control group; adjusted mean difference, 4.92 (95% CI 2.61-7.24;
Changes in knowledge using scores obtained with multiple-choice questions between blended-learning group and pure e-learning group.
Knowledge MCQa |
Blended-learning group |
Pure e-learning group |
Comparisons between two groups | |||
Baseline | Postcourse | Baseline | Postcourse | Adjusted differenceb, |
||
Course module 1c | 69.69 (68.10-71.27) | 93.21 (92.49-93.93) | 69.63 (68.16-71.1) | 88.29 (87.19-89.40) | 4.92 (2.61-7.24) | <.001 |
Course module 2d | 71.20 (69.75-72.65) | 94.05 (93.37-94.73) | 72.71 (71.38-74.05) | 90.22 (89.12-91.31) | 3.67 (1.17-6.18) | .004 |
Course module 3e | 74.12 (72.45-75.79) | 93.88 (93.08-94.68) | 73.85 (72.37-75.34) | 89.09 (87.89-90.30) | 4.63 (2.12-7.14) | <.001 |
aMCQ: multiple-choice questions.
bAdjusted difference is the mean difference between groups (intervention-control) adjusted for time of assessment and intervention × time interaction in a multilevel model with township center and participants as a random effect.
cCourse module 1: health management of patients with hypertension.
dCourse module 2: health records management for residents.
eCourse module 3: vaccination.
A questionnaire response rate of 71.9% (409/569) was achieved in the blended-learning group compared with 80.3% (452/563) in the pure e-learning group. Trainees' subjective opinions toward the interventions were investigated, including training benefits (confidence increase, aim realization, and knowledge improvement), changes in learning interest, and satisfaction with the training mode and the interaction. A majority of PHCWs agreed that the contents were well relevant to their work (93.9% in experimental group vs 94.5% in control group,
Questionnaire evaluation of the training between the blended-learning and pure e-learning group.
Courses evaluation questions | Blended-learning group (N=409) |
Pure e-learning group (N=452) |
ORb |
|||||||||
1 | 2 |
3 | 4 | 5 | 1 | 2 | 3 | 4 | 5 | |||
1. The courses are relevant to the daily work. | 157 (38.7) | 224 (55.2) | 15 (3.7) | 5 (1.2) | 5 (1.2) | 169 (37.4) | 258 (57.1) | 24 (5.3) | 1 (0.2) | 0 (0) | 0.89 (0.50-1.58) | .70 |
2. I achieved the objectives of each course. | 141 (34.5) | 228 (55.8) | 30 (7.3) | 9 (2.2) | 1 (0.3) | 139 (30.8) | 245 (54.2) | 64 (14.2) | 4 (0.9) | 0 (0) | 1.63 (1.08-2.48) | .02 |
3. Participation in the training had increased my interest in learning. | 153 (37.5) | 216 (53.0) | 32 (7.8) | 5 (1.2) | 3 (0.5) | 147 (32.5) | 231 (51.1) | 61 (13.5) | 12 (2.7) | 1 (0.2) | 1.85 (1.22-2.80) | .003 |
4. Participation in the training had increased my confidence. | 150 (36.9) | 211 (51.8) | 31 (7.6) | 14 (3.4) | 1 (0.3) | 159 (35.2) | 230 (50.8) | 43 (9.5) | 19 (4.2) | 1 (0.2) | 1.27 (0.85-1.91) | .25 |
5. Participation in this training had improved my knowledge. | 183 (45.5) | 199 (49.5) | 12 (3.0) | 8 (2.0) | 0 (0) | 191 (42.3) | 243 (53.8) | 16 (3.5) | 2 (0.4) | 0 (0) | 0.79 (0.42-1.52) | .48 |
6. Participation in the training increased the interaction with others. | 155 (38.1) | 206 (50.6) | 35 (8.6) | 11 (2.7) | 0 (0) | 144 (31.9) | 226 (49.8) | 70 (15.7) | 12 (2.6) | 0 (0) | 1.77 (1.20-2.60) | .004 |
7. I would like to try the training mode again. | 190 (46.6) | 187 (45.8) | 20 (4.9) | 4 (1.0) | 7 (1.7) | 205 (45.4) | 205 (45.4) | 36 (8.0) | 5 (1.1) | 1 (0.2) | 1.25 (0.77-2.02) | .37 |
8. Overall, I was satisfied with the training experience. | 195 (48.3) | 181 (44.8) | 23 (5.7) | 2 (0.5) | 3 (0.7) | 136 (30.1) | 263 (58.2) | 49 (10.8) | 4 (0.9) | 0 (0) | 1.78 (1.11-2.88) | .02 |
aResponses to questions about the feedback on Web-based training platform were on a 5-point Likert scale, ranging from 1 (strongly agree) to 5 (strongly disagree).
bOR: odds ratio.
cUnivariate logistic regression analysis was used to compare the differences between two groups (dependent variable as two categories with combining scores 1, 2 into one category and scores 3, 4, 5 to another category).
This study suggested that in rural China, a blended approach to BPHS training was more effective in improving knowledge than a pure e-learning approach. Trainees in blended-learning group expressed a higher satisfaction level about their learning experiences than pure e-learning trainees. Our study demonstrates the feasibility of applying Internet-related technology to PHCWs’ training on BPHS and explores the various training modes to improve the knowledge of PHCWs in rural China.
Currently, the inequalities in health care provision between urban and rural areas and the inequalities in the distribution of health workers remain serious problems in China [
Our study suggests that the blended-learning approach is more effective than pure e-learning in terms of knowledge achievement. This is supported by a recent meta-analysis of 56 studies finding that blended learning appears to be more effective than or at least as effective as e-learning [
In our study, we found that blended-learning trainees had a higher satisfaction level about their learning experiences than pure e-learning trainees. As for the case-based problem solving courses, social and collaborative learning experiences are important to help individuals in thinking, learning, and finding a solution for problems [
Consideration of learning outcome alongside the devoted costs and resources was important for educators to effectively review the educational interventions [
Although blended learning shows positive learning outcomes and satisfaction level in the study, barriers to the implementation among PHCWs still exist. Possible barriers to blended learning include technical difficulties, such as interrupted or limited Internet connection, poor computer literacy, and hindrance in accessing learning resource material, as reported previously [
To our knowledge, this is the first study to compare the effectiveness of a blended-learning approach with a pure e-learning approach to BPHS training among rural PHCWs. The main strengths of this study include the relatively large sample size, and both subjective and objective evaluation methods applied for comparison. Furthermore, our study provided more evidence on the effects of blended learning in comparison with pure e-learning.
The study has five limitations. First, the dropouts in both groups were seen in our study. The dropout rates were similar in the blended group and pure e-learning group in course module 1 (18.5% vs 15.5%) and course module 2 (16.7% vs 14.9%). As for course module 3, we should caution that the dropout rate was differentially higher in the blended group (21.8%) than that in the pure e-learning group (13.9%). However, the background characteristics of participants who drop out in two groups in the course module 3 were similar. As well, there were no significant differences in the comparison of background characteristics between dropouts and non-dropouts. In addition, among dropouts who had completed the baseline assessment of course module 3, there was no baseline difference between those who completed (mean score 73.98 [SD 17.26]; n=930) and dropouts (mean score 72.29 [SD 23.75]; n=155) with difference, 1.69 (95% CI −1.43 to 4.81;
In conclusion, blended approaches to BPHS training resulted in a better knowledge achievement and a higher satisfaction level than pure e-learning approaches among PHCWs in rural China. Using more effective training modes to improve PHCWs’ knowledge on BPHS can help enhance the PHCWs’ competency and accordingly improve the quality of health care in rural China in order to achieve health equity. To provide more rigorous evidence on the effects of blended learning in comparison with pure e-learning, more research is needed in the future.
basic public health services
Center for Disease Control and Prevention
electronic learning
gross domestic product
multiple-choice questions
odds ratio
primary health care workers
tuberculosis
The authors thank all of the participants for their contributions to this study. This study was supported by the project “Strengthening Primary Health Care Workers’ Competence by Using an Internet-Based Interactive Platform in Rural China” funded by the Ministry of Science and Technology of China. The funder had no involvement in the design, analysis, or reporting of the study.
WRY conceptualized and designed the study. XXZ, ZXZ, FS, and QL performed the study. XXZ and ZXZ analyzed the data. XXZ prepared the first draft of the paper. All authors contributed to the revision of the manuscript.
None declared.