This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Electronic learning (e-learning) through mobile technology represents a novel way to teach emergent otorhinolaryngology-head and neck surgery (ORL-HNS) disorders to undergraduate medical students. Whether a cognitive style of education combined with learning modules can impact learning outcomes and satisfaction in millennial medical students is unknown.
The aim of this study was to assess the impact of cognitive styles and learning modules using mobile e-learning on knowledge gain, competence gain, and satisfaction for emergent ORL-HNS disorders.
This randomized controlled trial included 60 undergraduate medical students who were novices in ORL-HNS at an academic teaching hospital. The cognitive style of the participants was assessed using the group embedded figures test. The students were randomly assigned (1:1) to a novel interactive multimedia (IM) group and conventional Microsoft PowerPoint show (PPS) group matched by age, sex, and cognitive style. The content for the gamified IM module was derived from and corresponded to the textbook-based learning material of the PPS module (video lectures). The participants were unblinded and used fully automated courseware containing the IM or PPS module on a 7-inch tablet for 100 min. Knowledge and competence were assessed using multiple-choice questions and multimedia situation tests, respectively. Each participant also rated their global satisfaction.
All of the participants (median age 23 years, range 22-26 years; 36 males and 24 females) received the intended intervention after randomization. Overall, the participants had significant gains in knowledge (median 50%, interquartile range [IQR]=17%-80%,
Mobile e-learning is an effective modality to improve knowledge of emergent ORL-HNS in millennial undergraduate medical students. Our findings suggest the necessity of developing various modules for undergraduate medical students with different cognitive styles.
Clinicaltrials.gov NCT02971735; https://clinicaltrials.gov/ct2/show/NCT02971735 (Archived by WebCite at http://www.webcitation.org/6waoOpCEV)
The large investment involved in undergraduate medical education (UME) for students and society has led medical schools worldwide to seek strategies and methods to improve their students’ progress [
Certain learning characteristics have been positively correlated with academic success; for instance, strong motivation and enjoying studying have been identified as positive predictors during the undergraduate year of medical school [
Mobile technology has gained popularity in recent years as a means of immediate interactive multimedia (IM) communication and to access the Internet. Embedded e-learning in a smartphone or tablet can affect the educational environment. In this context, it has been termed “mobile technology in e-learning (M-TEL),” and it has been reported to represent the next natural frontier in the evolution of e-learning [
Reducing training time can limit the number of topics taught in an UME curriculum, including otorhinolaryngology-head and neck surgery (ORL-HNS). However, at least 20% of primary care complaints are related to ORL-HNS, and a substantial downstream effect on managing ORL-HNS problems has been reported in family medical practice [
We conducted this prospective study from August 1, 2015 to July 31, 2017 at a university (Department of ORL-HNS, Faculty of Medicine, Chang Gung University, Taoyuan, Taiwan). This study included two parts: (1) pilot system-design study, and (2) validation study. This study was approved by the institutional review board of Chang Gung Medical Foundation (No: 105-5290C), and all procedures were conducted in compliance with the Declaration of Helsinki 1975. The participants were informed about the aims of the study, and written informed consent was obtained from all participants. The study proposal was registered at ClinicalTrials.gov (NCT02971735).
In the pilot system-design study [
Using the IM module, the learners could operate a leading character to search for and interact with other nonplayer characters to procure instructional materials, to review acquired instructional slides (maximal 80), and to win five small game-based quizzes (
In the PPS module, the learners chose and watched 10 visual-auditory text-image videos of emergent ORL-HNS disorders (a total of 80 min) by themselves. Video lectures were created by recording Microsoft PowerPoint presentations with audio narrations, timings, and ink gestures using Camtasia Studio software version 8 (TechSmith, Okemos, MI, USA). The learners were free to watch the videos at any time, and they could also rewind and fast-forward the videos as needed (
Analysis, design, development, implementation, and evaluation (ADDIE) model for designing effective instruction of mobile technology in electronic learning (e-learning).
During the course, all of the participants arbitrarily reviewed 80 slides by themselves. The instructional content of the two learning modules was confirmed to be correlated and equivalent by 2 investigators from the study team (
The validation study was a prospective, parallel-controlled, randomized clinical trial assessing the impact of cognitive styles and learning modules using M-TEL on knowledge gain, competence gain, satisfaction, and learning experience.
A total of 60 consecutive volunteers were recruited from a teaching clinic for the validation study from November 23, 2016 to July 5, 2017. All of the volunteers had at least a basic level of computer literacy, and they were shown the practical aspects of using tablets and apps. The inclusion criteria were as follows: (1) age >20 years; and (2) undergraduate medical students (clerkship). The exclusion criteria were: (1) previous ORL-HNS training; and (2) declining to participate.
Start of the apps. Learners read the adventure story and objectives (story symbol), played four instructional domains (red arrow symbol), reviewed instructional materials (book symbol), assessed learning progress (bar chart symbol), and got the helps (hint symbol) on the start screen.
Screenshots of the interactive multimedia module. Learners arbitrarily operated a leading character to run, jump, and interact with other nonplayer characters (up) to procure instructional materials (middle). After a small session, learners need to complete small game-based quizzes (low).
There were four different face-to-face assessments. The cognitive style of the participants was assessed using the 25-item GEFT after enrollment [
Pretests including a 15-min 10-question standard MCQs to evaluate the students’ existing knowledge (range 0-100) and a 15-min 5-question MSTs to assess their existing competence (range 0-100) with regard to “emergent ORL-HNS disorders” were given to the students. Each textbook-based MCQ was designed to be answered within 90 seconds and was preselected according to the results of item analysis. The MSTs presented the learners with written descriptions of five scenarios with or without images/videos and asked them to select the appropriate responses from 5 MCQs for one emergent ORL-HNS disorder using the methodology described in a previous publication of key features approach [
The students were then asked to complete a global satisfaction score (GSS) (range 0-10) questionnaire.
Blinding to the purpose of the study during recruitment was maintained to minimize preparation bias. After the participants had provided consent and completed the GEFT and pretests, we randomly assigned them (1:1) to the IM group and PPS group (
After randomization, the participants were unblinded and used fully automated courseware containing IM or PPS module on a 7-inch tablet in an ordinary office environment for 100 min. Before using the courseware, the functionality of the tablet was explained to the participants. The IM group participated in a parkour course to find and read the instructional materials and played small quiz games that were different from the MCQ and MST questions. The students in the PPS group used an app to read and listen to instructional materials in 10 linear-designed sessions. After completing the brief sessions, the IM and PPS learners could review simple slides of the instructional materials.
The primary outcome measure was the percentage change in MCQ score (ie, “knowledge gain”) after the M-TEL. Other outcomes were the percentage changes in MST (ie, “competence gain”) and GSS.
A priori sample size was estimated using primary outcome effects (percentage change in MCQ score) based on a pilot study (IM module: 43% [SD 18%]; PPS module: 35% [SD 21%]). A two-tailed Wilcoxon signed-rank test to calculate the sample size of 26 in each group (normal parent distribution; calculated effect size: 0.41; type I error: 0.05; power: 80%). Assuming a 10% dropout rate to fulfill the criteria of intention-to-treat analysis, we needed at least 29 participants in each group. Accordingly, we decided to enroll a total of 60 students to show the difference in percentage change in MCQ score.
Because the primary outcome measure (percentage change in MCQ score) was not normally distributed according to the D’Agostino-Pearson omnibus normality test, percentage changes ([after value-before value]/[before value] × 100) in MCQ, MST, and GSS; and AttrakDiff2 scores were compared between groups using the Wilcoxon signed-rank test, Mann-Whitney
Screenshots of the PowerPoint Show module. Learners watched 10 visual-auditory text-image videos of emergent otorhinolaryngology-head and neck surgery (ORL-HNS) disorders. The instructional slides of this module were identical to those of the interactive multimedia module and arranged linearly.
The Consolidated Standards of Reporting Trials flow diagram.
A total of 60 undergraduate medical students were screened, all of whom (median age 23 years, range 22-26 years; 36 males, 60% and 24 females , 40%) were randomized 1:1 to the IM group or PPS group as shown in the Consolidated Standards of Reporting Trials flow diagram (
Overall, all of the participants showed significant improvements in MCQ score (
Using the original definition of FD and FI defined by Witkin [
Demographics, cognitive style, learning outcomes, satisfaction, and experience.
Variables | Overall, N=60 | Interactive multimedia group, N=30 | PowerPoint show group, N=30 | ||
Age in years, median (IQRb) | 23 (23-24) | 23 (23-24) | 23 (23-24) | .21 | |
Male sex, n (%) | 36 (60) | 20 (67) | 16 (53) | .43 | |
Group embedded figures test score, median (IQR) | 17 (15-18) | 18 (15-18) | 17 (16-18) | .78 | |
Field-dependence, n (%) | 5 (8) | 3 (10) | 2 (7) | >.99 | |
Multiple-choice questions-before, median (IQR) | 40 (40-50)c | 40 (40-60)c | 45 (30-50)c | .47 | |
Multiple-choice questions-after, median (IQR) | 70 (60-80)c | 70 (58-70)c | 70 (60-80)c | .72 | |
Percentage change in multiple-choice questions, median (IQR) | 50 (17-80)d | 40 (13-76)d | 60 (20-100)d | .42 | |
Multimedia situational test-before, median (IQR) | 80 (60-80)c | 80 (60-80) | 80 (60-80)c | .84 | |
Multimedia situational test-after, median (IQR) | 80 (80-100)c | 80 (60-80) | 80 (80-100)c | .003 | |
Percentage change in multimedia situational test, median (IQR) | 13 (0-33)d | 0 (−21 to 38) | 25 (0-33)d | .16 | |
Global satisfaction score, median (IQR) | 7 (5-9)d | 8 (6-9)d | 6 (4-7) | .01 |
aMann-Whiney
bIQR: interquartile range.
c
d
Comparisons of demographics, learning model, outcomes, satisfaction, and experience between classical cognitive styles.
Variables | Classical field-dependent, N=5 | Classical field-independent, N=55 | ||
Age in years, median (IQRb) | 23 (22-24) | 23 (23-24) | .45 | |
Male sex, n (%) | 2 (40) | 34 (62) | .38 | |
Group embedded figures test score, median (IQR) | 9 (4-12) | 18 (17-18) | <.001 | |
Interactive multimedia, n (%) | 3 (60) | 28 (51) | >.99 | |
Multiple-choice questions-before, median (IQR) | 40 (25-60) | 40 (40-50)c | .53 | |
Multiple-choice questions-after, median (IQR) | 60 (50-80) | 70 (60-80)c | .70 | |
Percentage change in multiple-choice question, median (IQR) | 67 (−7 to 200) | 50 (17-80)d | .90 | |
Multimedia situational test-before, median (IQR) | 80 (50-100) | 80 (60-80)c | .63 | |
Multimedia situational test-after, median (IQR) | 80 (70-90) | 80 (80-100)c | .92 | |
Percentage change in multimedia situational test, median (IQR) | 0 (−30 to 92) | 25 (0-33)d | .68 | |
Global satisfaction score, median (IQR) | 6 (4-7) | 7 (5-9)d | .25 |
aMann-Whiney
bIQR: interquartile range.
c
d
In this study, most of the participants were categorized as classical FI learners, and we were unable to determine which FI learners were more suitable for M-TEL using classical classification [
Comparisons of demographics, learning models, outcomes, satisfaction, and experience among modified cognitive styles.
Variables | Modified field-dependent, N=15 | Modified field-intermediate, N=17 | Modified field-independent, N=28 | ||
Age in years, median (IQRb) | 23 (22-24) | 23 (23-24) | 23 (23-24) | .74 | |
Male sex, n (%) | 8 (53) | 12 (71) | 16 (57) | .56 | |
Group embedded figures test score, median (IQR) | 14 (12-15) | 17 (17-17) | 18 | <.001 | |
Interactive multimedia, n (%) | 8 (53) | 7 (41) | 15 (54) | .69 | |
Multiple-choice questions-before, median (IQR) | 50 (30-60)c | 40 (40-50)c | 40 (40-50)c | .47 | |
Multiple-choice questions-after, median (IQR) | 70 (60-80)c | 70 (65-80)c | 70 (53-70)c | .48 | |
Percentage change in multiple-choice question, median (IQR) | 40 (17-100)d | 75 (33-100)d | 45 (15-75)d | .34 | |
Multimedia situational test-before, median (IQR) | 80 (60-80)c | 80 (60-80)c | 70 (60-80)c | .74 | |
Multimedia situational test-after, median (IQR) | 80 (80-80)c | 80 (70-100)c | 80 (63-95)c | .83 | |
Percentage change in multimedia situational test, median (IQR) | 25 (0-33)d | 25 (0-29)d | 0.0 (0-37)d | .82 | |
Global satisfaction score, median (IQR) | 6 (3-7) | 8 (7-10)d | 7 (5-8)d | .02 |
aMann-Whiney
bIQR: interquartile range.
c
d
Comparisons of the effect of modified cognitive style and module of mobile technology in electronic learning on outcomes.
Outcomesa | Test statistics | Standard error | Standard test statistics | Adjusted |
||
Modified cognitive style–learning module | −0.53 | 0.18 | −2.92 | .003 | .01 | |
Modified cognitive style–percentage change | 0.71 | 0.18 | −3.88 | <.001 | <.001 | |
Learning module–percentage change | −1.24 | 0.18 | −6.80 | <.001 | <.001 | |
Modified cognitive style–learning module | −0.57 | 0.18 | −3.10 | .002 | .006 | |
Modified cognitive style–percentage change | 0.11 | 0.18 | 0.59 | .55 | >.99 | |
Learning module–percentage change | −0.46 | 0.18 | 2.51 | .012 | .04 | |
− | ||||||
Modified cognitive style–learning module | −0.52 | 0.18 | −2.83 | .005 | .01 | |
Modified cognitive style–global satisfaction score | −1.19 | 0.18 | −6.53 | <.001 | <.001 | |
Learning module–global satisfaction score | −1.71 | 0.18 | −9.36 | <.001 | <.001 |
aFriedman’s two way analysis of variance test.
The main findings of this study are that M-TEL outside the classroom can help undergraduate medical students to strengthen their knowledge and competence of emergent ORL-HNS disorders, and to provide an enjoyable learning experience overall. In addition, our findings suggest that millennials can significantly gain knowledge rather than reinforce competence using an IM module. Despite the similar efficacy of both modules, the students preferred the IM module to the PPS module because of it being more efficient and enjoyable to use. Although the classical classification of cognitive style [
There are several limitations to this randomized controlled trial. First, this study was quasi-experimental because of the lack of probability sampling. Even though we selected individuals based on their availability to the investigators, the sample size was representative of the target population (>50% were classmates). Second, the study used different posttest questions to measure learning outcomes, and the interaction between taking a pretest and the intervention itself may threaten the external validity. A design which does not use a pretest would have been preferable [
As mentioned above, both the IM and PPS modules used the same textbook-based learning material. However, the IM module applied game-design elements and game principles in nongame contexts (gamification) [
Teaching and learning processes are thought to be affected by various cognitive variables. If medical students receive training, which has been designed according to their individual needs, they may develop a sense of competence and positive self-perception [
Medical teachers in Taiwan have traditionally assumed that medical students can automatically adapt to the instructional modality and material by themselves to learn a topic, regardless of whether or not they are effective. Although we previously postulated that both learning module and cognitive style could affect competence gain for UME, we found the undergraduate medical students using the PPS learning module had significantly higher improvements in competence, whereas there was no change in competence in the IM learners. Even though the IM module was more attractive and satisfactory than the PPS module, more integrated formats of instructional material were needed to reduce extraneous cognitive load to facilitate problem-solving performance [
Since learners can start and stop M-TEL at any time or place of their choosing [
M-TEL using conventional PPS and novel IM modules seems to be an effective method to teach emergent ORL-HNS disorders to undergraduate medical students. The PPS module represented a formal, serious learning modality, whereas the IM module represented a satisfactory, enjoyable way for the millennial students to learn. Cognitive style and M-TEL module significantly affected knowledge gain and satisfaction, and the modified FINT learners had the highest gains in knowledge and satisfaction when using the IM module. These findings support the development of M-TEL, including various learning modules for undergraduate medical students with different cognitive styles.
CONSORT-EHEALTH checklist V 1.6.1.
field-dependent
field-independent
field-intermediate
group embedded figure test
global satisfaction score
interactive multimedia
interquartile range
multiple-choice questions
multimedia situational test
mobile technology in e-learning
otorhinolaryngology-head and neck surgery
PowerPoint show
undergraduate medical education
This study was financially supported by grants from the Ministry of Science and Technology, Taiwan, ROC (104-2511-S-182-010 & 105-2511-S-182A-006) and a grant from the Chang Gung Medical Foundation, Taiwan, ROC (CMRPG3F1091). The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.
We are indebted to our trained study staff, especially Chung-Fang Hsiao, Hsiu-Yen Yu, Cheng-Hao Yang, Hao-Xiang Zhang, and Ming-Xuan Chen.
LAL, YPC, SLW, CKC, and HYL participated in the conception and design of this work. LAL, YPC, CJK, LJH, WNL, TJF, and HYL collected data. LAL, YPC, CGH, JTF, SLW, CKC, and HYL analyzed and interpreted data. LAL, YPC, SLW, CJK, LJH, WNL, TJF, and HYL carried out the development of the project software. All authors participated in the writing of the manuscript and take public responsibility for it. LAL, YPC, CGH, JTF, SLW, CKC, and HYL revised it critically for important intellectual content. All authors reviewed the final version of the manuscript and approve it for publication. All authors attested to the validity and legitimacy of the data in the manuscript and agree to be named as authors of the manuscript.
None declared.