Technology Acceptance and Information System Success of a Mobile Electronic Platform for Nonphysician Clinical Students in Zambia: Prospective, Nonrandomized Intervention Study

Introduction

Background

Zambia, a country in south-central Africa, faces significant challenges in its overall disease burden and scant numbers of health workers across all sectors and rural regions in particular [1-3]. The current severe shortage of Zambian health workers and their skewed distribution toward urban areas underline the dire need for an upscale of skilled and knowledgeable health workers, especially in rural areas. The ratio of health workers in Zambia compared with the population is 1.2 per 1000 people, which is far from the World Health Organization Sustainable Development Goals index threshold of 4.45 health workers per 1000 people [2,4]. In a country with over 750,000 km² area, Zambia is sparsely populated by approximately 20 inhabitants per square kilometer [5], which further aggravates the dense coverage of and proximate access to health care. Zambia’s rural areas are severely underserved with only 7 clinicians per 10,000 people (urban areas: 16/10,000 people) and are insufficiently covered by health facilities. The recent Zambian National Health Strategic Plan [1] points out that at times “the situation is so severe that some facilities in rural areas have insignificant numbers of staff and in the worst scenario are managed by unqualified staff.” In 2002, to address the need for better-qualified nonphysician clinicians to cover a broader scope of health care than that provided by clinical officers, Zambia started training associate physicians at the Chainama College of Health Sciences (CCHS), Lusaka, to be known as medical licentiate practitioners (MLPs) [6]. MLPs manage advanced and common medical conditions and cover competencies such as surgical care, cesarean sections, and management of complex treatments. Most MLPs work at rural district hospitals or peripheral health centers. Retention of MLPs is high, and over 270 MLPs work in the Zambian health system, thus strengthening health care in rural areas [1]. As a result, the Ministry of Health has set an objective to more than double the MLP workforce to 600 by 2025 [1]. Although the number of MLPs trained at CCHS has increased, especially in rural areas, CCHS as the main national training institution for MLPs has experienced restricted resources and infrastructure to adequately upscale the sorely needed increase in the number of students.

Research Objectives

To strengthen the quantity and quality of training for MLPs, we introduced a blended learning approach that includes a self-directed electronic health platform (e-platform) [7]. The primary objective of the e-platform is to alleviate the shortcomings of the medical licentiate program training caused by shortages of medical lecturers and learning materials in MLP training sites. The e-platform covers 2 components: (1) electronic learning (e-learning) for medical education with Web-based and offline static and interactive learning materials including lecture presentations, books, virtual patient cases, pictures, and videos and (2) health care practice support with medical treatment guidelines and algorithms to diagnose and treat patients. Details on e-platform contents are described elsewhere [8,9]. MLP students were provided with 7-inch tablets preloaded with offline contents of the e-platform through the Moodle mobile app [10]. The blended learning approach was piloted from January 2016 until August 2016 [8], and the evaluation of the first year of full implementation of the e-platform for the MLP was conducted from September 2016 to August 2017. The evaluation included both a qualitative [8,9] and quantitative methodology. The objective of the quantitative evaluation was to empirically explore the acceptance based on the technology acceptance model (TAM; TAM2 as a baseline model [11-13]) and information system (IS) success based on the IS success model (ISSM) [14] of the e-platform. The TAM assumes that the users’ acceptance of technology depends on 2 variables: (1) perceived usefulness and (2) perceived ease of use [15]. In the original TAM, these 2 variables served as a proxy to determine the users’ attitude toward using the technology [16]. In 2000, the extended TAM was proposed (TAM2) [17]to include social and cognitive influences to better reflect the complexity of the users’ technology acceptance decisions and the increased complexity of more developed ISs [15]. The ISSM is a measure for IS that is organized in 3 levels: (1) the first level includes ISSM quality constructs that impact the user’s satisfaction and intention to use the IS, (2) the second level examines the systems’ performance, and (3) the third level looks at net benefits of an IS.

With questionnaires based on the TAM2 and ISSM, we wanted to measure technology acceptance and IS success, respectively, answering the following specific research questions:

1. Is the technology of e-learning for medical education accepted, that is, do students and medical lecturers agree with the technological environment of e-learning as a mode of learning and teaching for the CCHS?
2. To which extent is medical e-learning used for learning and teaching by MLP students and medical lecturers and how is medical e-learning used Web-based and offline (context, frequency, materials and IS success respectively)?

Methods

Study Participants

During the study period from September 2016 to August 2017, a total of 83 students (3rd study year [SY] n=23, 4th SY n=32, and bridging SY n=28) were registered in the MLP at CCHS and together with the medical lecturers (n=36) constituted the total study population. All enrolled MLP students were eligible for the study, so no sample size specification was applicable. At the time of the study, the MLP student population included 3rd and 4th SY students and bridging SY students (those who had already acquired an MLP diploma in the past and continued 1 more year in the MLP to receive a Bachelor of Science degree).

Study Design and Procedures

The prospective, nonrandomized intervention study was conducted during the MLP SY 2016/17 that started in September 2016 and ended in July 2017. The study took place in the capital—Lusaka—and 10 rural hospitals were used as practical training sites for the MLP. The CCHS main campus, a medical multiprofessional training institution, was located about 10 km east of Lusaka’s city center and was one of the largest training institutions for Zambian health workers.

Learning Materials

Learning materials for the e-platform mainly comprised readily available materials provided by medical lecturers according to the MLP curriculum and materials customized for the e-platform. Access to e-platform materials was available offline via the tablets and Web-based, via the Moodle learning management system [10]. E-platform contents included static and interactive elements, such as lecture presentations (mostly Microsoft PowerPoint), medical books, interactive virtual patient cases, medical images, and short videos on medical procedures (see Table 1). Health care practice was supported with materials on standard treatment guidelines and algorithms for diagnosis and treatment.

Data Collection

Fieldwork for this study was conducted in collaboration with CCHS. In September 2016, study baseline data were collected before the intervention and included student demographic data and technology experience. Students’ name, date, place, date of birth, marital status, SY, prior studies, year of graduation, and medical experience in years were collected (see Table 2) together with a 10-item paper-based questionnaire about technology experience (see Multimedia Appendix 1). A questionnaire asked students about their usage and exposure to computers and mobile computing devices, level of comfort of internet navigation, and prior experience with and self-perceived usefulness of e-learning.

After completion of the baseline data, the MLP students were given tablets (7-inch and Android-based) preloaded with learning and clinical decision-making support materials and access to the Web-based e-platform. MLP lecturers were provided access to the Web-based MLP e-platform.

In August 2017, students’ data were collected on technology acceptance and IS success with paper-based questionnaires (5-item Likert scale: strongly disagree, disagree, neutral, agree, and strongly agree). The students’ technology acceptance questionnaire comprised 25 questions based on the TAM2 baseline model [11-13] with 6 constructs (see Table 3): (1) perceived ease of use, (2) perceived usefulness, (3) attitude, (4) behavioral intention, (5) self-efficacy, and (6) subjective norm; and 55 questions comprised DeLone and McLean’s IS success model [14] on the basis of 6 constructs (see Table 4): (1) information quality, (2) service quality, (3) system quality, (4) user satisfaction, (5) system use (the intention to use the system), and (6) net benefits. MLP students completed the questionnaires in small groups of approximately 30 students. The TAM-based questionnaire for medical lecturers included 45 questions. Medical lecturers were invited to complete the questionnaire via a cloud-based survey service within a 3-month time frame (July to October 2017), as they were dispersed throughout Zambia.

This study report conforms with the Consolidated Standards of Reporting Trials-Electronic Health [18] checklist (see Multimedia Appendix 2).

Objectives

The primary objective of the study was to evaluate the technology acceptance and IS success of the e-platform.

Statistical Methods

All data were cleaned in a systematic screening for completeness, plausibility, and consistency. Potential inconsistencies were resolved by checking against original data forms. Results were transferred into a spreadsheet and data were then analyzed with RStudio Desktop (open-source license version 1.1.463) [19].

For questionnaire items, we calculated the mean, standard deviation, and correlation coefficient r to see how well aligned questions were to the respective categories. Likert scale data were interpreted as interval scales [20], and data from various study participants were regarded as independent as we assumed that the behavior of one participant did not influence the behavior of another. Correlations of variables were explored with Kendall tau correlation coefficient as an estimator of correlation in the population.

Ethical Considerations

We communicated the purpose of this research to study participants and explained the study design and participation requirements. Before taking part in this study, all study participants agreed to and signed an informed consent explaining the study scope and purpose, and their right to withdraw at any point. Voluntary study participation was emphasized to be voluntary and assessments were solely part of this study. Furthermore, it was emphasized that the usage of the e-platform is tracked. The Biomedical Research Ethics Committee of the University of Zambia and the ethical committee of the University Hospital Heidelberg, Germany approved the study protocol.

Results

Demographics

Overall, the student population was predominantly male (58/74; 78%) with female students in the minority (16/74; 22%). The predominant age group was between 30 to 40 years (42/74; 57%; see Table 2). Most students had had more than five years of medical experience with an average of 10 years of working experience. Medical lecturers in the MLP were also predominantly male (10/14; 71%) with an average work experience of approximately 17 years and an average of 10 years of teaching experience.

Female students’ experience with technology was generally low, whereas males reported a predominantly midrange of technology experience (see Multimedia Appendix 3). Demographic details for age groups and medical and technology experience by gender are presented in Table 2.

Owing to a high number of missing data values, data from 9 MLP students were excluded from further analysis.

Technology Acceptance

With regard to technology acceptance, the highest rated categories by MLP students were Behavioral Intention (1.69; SD 0.58; r=0.67) and Attitude (1.72 SD=0.59; r=0.70), whereas Subjective Norm (1.92; SD=0.61; r=0.78) and Self-Efficacy (1.86; SD=0.69; r=0.65) were the categories with the lowest acceptance among students, followed by the categories of perceived usefulness (=1.85; SD=0.61; r=0.60) and perceived ease of use (=1.82; SD=0.61; r=0.61; see Table 3). The individual items with the highest agreement among MLP students were as follows: “For my future job, it is necessary to know how to use a computer” (item 21; 1.59; SD=0.55; r=0.77), “I am positive towards using a tablet for medical learning” (item 18; 1.60; SD=0.57; r=0.59), and “In my job, using a tablet is important” (item 13; 1.61; SD=0.52; r=0.64). The least agreement MLP students had were on the following items: “Using a tablet computer is compatible with all aspects of my studies” (item 10; 2.04; SD=0.78; r=0.64), “E-learning makes it easier to study course content” (item 9; 1.96; SD=0.62; r=0.49), and “Learning how to use the MLP e-learning platform is easy for me” (item 2; 1.96; SD=0.60; r=0.57).

Information System Success

For the IS success questionnaire, the category of Information Quality received the overall best ratings from MLP students (1.99; SD=0.80; r=0.57) followed by Net Benefits (2.00; SD=0.61; r=0.60) that contrasted with System Use with the lowest rating (2.47; SD=0.84; r=0.69) and Service Quality (2.29; SD=0.86; r=0.52) with the second lowest rating (see Tables 3 and 4). The specific items with the highest agreement among MLP students were all in the category of information quality, “Materials available on the MLP e-learning platform are useful to me” (item 3; 1.58; SD=0.56; r=0.49), “The MLP e-learning provides information relevant to MLP medical practice” (item 6; 1.63; SD=0.55; r=0.56), and “Materials on the MLP e-learning platform increase my quality of clinical care” (item 7; 1.74; SD=0.51; r=0.57). Items rated the lowest by MLP students were in the lowest-rated categories of system use and service quality, “I use the MLP e-learning platform daily several times” (item 32; 2.61; SD=0.90; r=0.76), “I depend upon the MLP e-learning platform” (item 17; 2.54; SD=0.92; r=0.38), and “The MLP e-learning platform provides rapid services” (item 33; 2.53; SD=0.82; r=0.72).

The Kendall tau variable correlation coefficients are shown in Tables 5 and 6 for the IS success model and TAM questionnaires, respectively. The bivariate relationship indicated that all of the variables (questionnaire items) were significantly correlated (correlations<0.05; see Tables 5 and 6).

For medical lecturers, the categories Perceived enjoyment (1.34; SD=0.48; r=0.73) and Perceived ease of use (1.86; SD=1.05; r=0.48) were rated the highest, whereas Behavioral intention (2.14; SD=0.83; r=0.54) and Self-efficacy (2.11; SD=1.19; r=0.72) were perceived as the categories with the least agreement (see Table 7). The Kendall tau variable correlation coefficients for the TAM questionnaire are shown in Table 8. Furthermore, the questions asked in addition to the TAM disclosed that 2 medical teachers had never used the e-platform (never n=2; often n=3; once n=1; several times n=1; frequently n=1) and 5 did not know how to access it. Overall, 50% (7/14) of medical lecturers stated that they had contributed to the e-platform with content or other e-learning–based activities, such as virtual patients (n=7). Correlation coefficients (Kendall ; nonparametric correlation) of the medical lecturers’ technology acceptance items indicated a correlation for most variables (see Table 8). High correlation significance (P<.01) was shown between the items perceived usefulness and attitude, perceived ease of use and self-efficacy, and perceived ease of use and perceived usefulness.

Discussion

Principal Findings

Overall, the questionnaires based on the TAM and IS success model indicated a positive reception of the e-platform and its usage with the offline-based tablets by MLP students and medical lecturers and identified strengths and shortcomings in the tablets and MLP e-platform. The questionnaires’ results elicited an overall acceptant attitude toward the e-platform and agreement with the technological environment of e-learning as a mode of learning and teaching by both, MLP students and medical lecturers.

The student study participants were quite heterogeneous in age, medical experience, and technological experience. The youngest student was aged 23 years and the oldest was 54 years. Some students had only 1 year of medical experience whereas others had accumulated over 28 years of medical experience. The same was applied to technological experience, as some students’ exposure to the MLP e-platform was the first time they had ever had a chance to experience continuous technology use for learning and accessing knowledge. Others, digital natives who had grown up with technologies, came into the MLP program equipped with 2 mobile devices and a laptop as an inherent part of their daily lives. Males seemed more confident in rating their self-perceived technology experience than females, who seemed more conservative in their self-assessment. Potentially, this finding may be the result of the prevailing gender-based stereotypes holding females back, as technology usage may not fit the stereotype of expectations for African females. In general, female students may be faced with “a subconscious or implicit bias against women scientists in higher education settings” as “stereotypes emerge early and continue to be salient throughout the lifespan” [21]. The implicit bias against women, which is unfortunately still prevalent, should be taken up and actively approached in the MLP e-platform’s implementation strategy. One study found, “for instance, college students were more likely to rate the same conference abstracts as lower in scientific quality if the author’s name was female instead of male, particularly if these topics had traditional masculine themes” [21]. Another study described a “moderating effect of gender difference in the adoption of multimedia technology for learning,” whereby males, “believe the technology can improve their performance when they find a fit between task and technology, and this belief would lead to their adoption of the technology” [22]. The study recommended (only for males) “that educators can better promote male students' adoption of multimedia technology by demonstrating how appropriate the multimedia technology given is for their learning” [22].

Age too can be a discriminative factor in technology acceptance and should be taken into consideration particularly for heterogeneous cohorts like the MLP students. “More attention should be paid to their support [elder MLP students] in the form of training, as well as meeting their unique personal needs” [23]. Overall, elder MLP students were more satisfied with the MLP e-platform than younger MLP students. The higher satisfaction may have been rooted in older students’ limited prior exposure to technology and consequently lower expectation levels. Thus, elder MLP students may have perceived the MLP e-platform as more satisfying than their younger colleagues.

In particular, MLP e-platform information quality and net benefits were perceived to be the most appealing characteristics based on the IS success model. A closer look at the individual questions of those 2 categories, however, shows that the ratings were more differentiated since students rated available materials as clear and useful, but questions about materials’ quality and sufficiency received the lowest overall scores (information quality). The quality and quantity of contents was perceived as a hindrance which may have been rooted in the short implementation period of the MLP e-platform in general which resulted in limited and rather static learning content that did not entirely reflect the MLP curriculum. Contents were initially composed only of lecture notes and adapted materials from a Malawian e-learning platform [24,25].

To this end, adopting a social media style for content creation and curation to allow for student-made, peer-reviewed content based on fellow student ratings, mixed with methods such as microlearning or learning nuggets, could potentially leverage the practicality of mobile devices (eg, tablets), and could be a significant next step for higher acceptance learning [26]. Curating available Web-based content may be of benefit to keep contents up to date while at the same time decreasing efforts to develop own learning content which constitutes a current major bottleneck—resource-intense and manual content creation. The MLP e-platform may then act also as a central hub for grouping and sharing reliable and appropriate information from the internet [27].

Students underlined with their high acceptance of the statements “For my future job, it is necessary to know how to use a computer,” “I am positive towards using a tablet for medical learning,” and “In my job, using a tablet is important” that they are willing and know the importance to make use of technologies, such as tablets, for their clinical work. Consequently, the categories of perceived usefulness denoting the relevance of the MLP e-platform toward job performance and perceived ease of use also had high agreement among MLP students. Similarly, for e-platform net benefits, the students found it useful in general, especially using the tablet in a clinical context for treating patients, but the e-platform did not support students well for their exams or help them achieve their learning tasks more efficiently. The subjective norm, a person's perception that most people who are important to him think he should or should not perform the behavior in question [28], was not rated as a strong factor compared with other questionnaire categories nor was e-platform self-efficacy, people's judgments of their capabilities to perform a given task [29]. The students’ low ratings for the subjective norm can be attributed to the e-platform still being a relatively new mode for the MLP. The MLP students did not find the tablet compatible with all aspects of their studies, nor did they perceive that the e-platform made studying easier. This shows a need for adaptation of the e-platform to better suit its users and enhance the overall ease of use. During the SY, tablet failure impacted the reliable usage of the offline e-platform (service quality). Most tablets failed either because of poor operating system upgrades or hardware fragility (for a more detailed description see the study by Barteit et al [9]). Furthermore, for some tablets, the battery life span reduced rapidly limiting its usage. Service quality and the quality of the materials, in turn, may have had an impact on the students’ low ratings for self-efficacy, which has been shown to have a direct and powerful effect on actual use over and above user intention [29].

Medical lecturers showed a general acceptance of technology for learning and teaching within the MLP program as the highest ratings for the questionnaires were given for perceived ease of use and perceived usefulness of the e-platform. However, most medical lecturers rated their colleagues’ attitude toward the MLP e-platform as not very positive, which indicates that many did not count on digitized methods to be a part of their teaching and potentially do not see these methods as valid instruments. Their perception toward the e-platform to improve MLP student medical performance received low agreement scores, which further underlines skepticism toward e-learning as a teaching method. In addition, medical lecturers conveyed low confidence toward using the e-platform for medical teaching in combination with a potentially conservative preference for textbook-based learning and lecture-style teaching as a core teaching method. The item, The use of the ML e-learning platform is better as compared to textbook learning, received one of the lowest ratings of all questions, and the categories of behavioral intention and self-efficacy of the e-platform both received very low scores indicating low agreement among medical lecturers. These low ratings further manifest a certain reluctance to employ the e-platform for teaching. Potentially, a bias was rooted in social desirability which influenced MLP lecturers’ answers to the questionnaire [30] as they knew that they were expected to show a certain openness to technologies. Thus, the results of this evaluation suggest a low medical lecturer uptake of the e-platform, which potentially resulted in limited learning materials and restricted integration of the e-platform in day-to-day teaching and training [9,31]. One MLP medical lecturer suggested increasing the allowance for site consultants so as to motivate them to make use of the e-platform. In fact, an increase in lecturer payment in Mexico showed no benefit, as it did not translate into an impact on student learning nor did it hold as a strategy for better teacher engagement in the digital learning and teaching process [32]. A more promising approach seems to be to consider the transformation of the teaching-learning process that entails a global approval and acceptance of teachers and the administration of digital technologies. Technology cannot fix outdated processes and substitute for well-qualified, motivated teachers. Thus, it may be of benefit to further educate and support teachers to use digital technologies for teaching and training [31], including content creation and e-learning didactics, such as flipped classrooms, adaptive learning [33], video-making with low-entry technologies such as smartphones, short mobile learning sessions (microlearning for on-the-job learning), and fostering technology-based social learning activities such as discussion and learning groups. An increased institutional presence may also improve lecturer approval and acceptance, such as an e-learning lab, which may take the physical shape of a dedicated part of the library with a few computers, where medical lecturers can seek technical support for making use of the e-platform or to prepare e-platform contents. Attitude has been identified as an important factor contributing to technology acceptance and may well change over time when more technological experience is gained [34]. Training teachers in digital literacy is not only of benefit for catering for the teaching-learning process but also may provide the cobenefit of preparing them to handle and embrace technologies in the medical field, which are pervasive and rapidly increasing as the digitization of medicine progresses [35,36].

Overall, medical lecturers did show a readiness and acceptance toward technology, since the majority negated that technologies were only for young people and agreed that they intended to use the e-platform to assist their medical teaching and that the e-platform could be helpful for their teaching.

Limitations

The limitations of the study are as follows:

• Ensuring questions are clear and not misleading: (1) Getting respondents to answer questions thoughtfully and honestly and (2) obtaining a sufficient number of completed questionnaires to enable meaningful analyses [37].
• No comparison group: All study participants were subject to the same intervention, so it was difficult to attribute the change in outcome to the introduction of the program with any certainty.

Conclusions

MLP students and medical lecturers accepted the e-platform as a method for medical teaching and learning and the evaluation of the MLP e-platform with its offline tablet-based component, which proved to be a feasible approach for teaching and learning within the low-resource environment at the CCHS in Zambia. This overall positive acceptance toward the e-platform constitutes a fertile base to scale up and implement the e-platform as a serious learning and teaching methodology within the blended learning strategy for the MLP program and beyond. Main shortcomings comprise the low uptake of the e-platform in everyday teaching and the scope and level of interactivity of the e-platform contents. The e-platform has the potential to be a cornerstone in the expansion of scaling up the training of urgently needed medical licentiates in Zambia, but it requires a profound transformation of the teaching and learning process that at its core is manifested in the curriculum and ongoing technology training for medical lecturers. From a technological standpoint, the tablet-based e-platform is a flexible tool that allows for electronic assessment, skill-based learning sessions, a digital logbook for the MLP, and an enhanced clinical decision-support system that could be incorporated in the continuous medical training for MLP graduates in rural health facilities. With increasingly affordable and viable technologies being available, the e-platform may potentially become a game changer in future medical education, especially in a low-resource context such as Zambia.