This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Smartphone-based learning, or mobile learning (m-learning), has become a popular learning-and-teaching strategy in educational environments. Blended learning combines strategies such as m-learning with conventional learning to offer continuous training, anytime and anywhere, via innovative learning activities.
The main aim of this work was to examine the short-term (ie, 2-week) effects of a blended learning method using traditional materials plus a mobile app—the iPOT mobile learning app—on knowledge, motivation, mood state, and satisfaction among undergraduate students enrolled in a health science first-degree program.
The study was designed as a two-armed, prospective, single-blind, randomized controlled trial. Subjects who met the inclusion criteria were randomly assigned to either the intervention group (ie, blended learning involving traditional lectures plus m-learning via the use of the iPOT app) or the control group (ie, traditional on-site learning). For both groups, the educational program involved 13 lessons on basic health science. The iPOT app is a hybrid, multiplatform (ie, iOS and Android) smartphone app with an interactive teacher-student interface. Outcomes were measured via multiple-choice questions (ie, knowledge), the Instructional Materials Motivation Survey (ie, motivation), the Profile of Mood States scale (ie, mood state), and Likert-type questionnaires (ie, satisfaction and linguistic competence).
A total of 99 students were enrolled, with 49 (49%) in the intervention group and 50 (51%) in the control group. No difference was seen between the two groups in terms of theoretical knowledge gain (
The blended learning method led to significant improvements in motivation, mood state, and satisfaction compared to traditional teaching, and elicited statements of subjective improvement in terms of competence in English.
ClinicalTrials.gov NCT03335397; https://clinicaltrials.gov/ct2/show/NCT03335397
Smartphone-based learning or mobile learning (m-learning), a term that highlights the type of device used, has become a popular learning-and-teaching strategy. It has been defined as the ability to access educational resources, tools, and materials anytime and from anywhere, using a mobile device, such as a smartphone [
M-learning is being increasingly used by undergraduate and postgraduate students undertaking specialty or continuous training in the health sciences [
Motivation is an important factor in learning and performance [
New technologies may, however, also have a negative effect on learning. A recent study [
Blended learning that combines m-learning with conventional learning to seek the benefits of both [
This study was a two-arm, prospective, single-blind, randomized controlled trial (ClinicalTrials.gov identifier: NCT03335397). The study subjects were 99 students in their first year of a health sciences degree at the Faculty of Health Sciences, University of Granada, Spain.
The sample size calculation was based on a previous study [
Subjects were recruited via a talk given on the first day of the course unit entitled Basic Health Science, a unit lasting one semester. It was emphasized that no assessment made during the trial would have any effect on final grades and that participation was totally voluntary. To be included, the subjects had to (1) possess basic skills in handling mobile apps, (2) have a smartphone running either Android operating system (OS) or iOS software, (3) install the iPOT mobile learning app, (4) be enrolled in the above course unit, and (5) have a basic knowledge of English, accredited or not. In addition, all subjects had to provide informed consent to be included. Students repeating the unit were excluded.
Subjects who met the inclusion criteria were randomly assigned to either the intervention group (ie, blended learning, involving traditional learning plus m-learning via the use of the iPOT app) or the control group (ie, traditional on-site learning).
Flow diagram of the recruitment and randomization process. OS: operation system.
This study was conducted over 2 weeks in the first semester of the 2017-2018 academic year. For both groups, the educational program involved 13 lessons on basic health science. The subjects of both groups were progressively introduced to conventional learning materials (ie, books, PowerPoint presentations, and journals available in the university library). The subjects in the intervention group, however, also received the iPOT app to reinforce the educational program; it provided no additional material. Both the intervention and control subjects were free to reinforce the taught information using any of the traditional sources available. All subjects in the intervention group received a QR (Quick Response) code to provide them access to the Apple App Store [
M-learning lessons were enabled in the app over the last 2 weeks of the teaching period, allowing the intervention group subjects to review the information previously taught. Students were instructed to review each lesson via the six learning modalities available—quizzes (multiple-choice questions), alphabet soup (word search), hangman, hieroglyphics, puzzles (word sorting), and word-coupling activities—and were encouraged to use them; differences in use time were accounted for (see The iPOT Mobile Learning App: Internal Design and Technical Specifications section). It is important to highlight that during these 2 weeks, the control group subjects were encouraged to review the taught material via the above-mentioned books, PowerPoint presentations, and journals.
The iPOT app was designed by a research group composed of lecturers in health science and an internship student of computer engineering, all at the University of Granada. The app is a hybrid, multiplatform (iOS and Android), smartphone app with an interactive teacher-student interface (see
Top-level view of the iPOT app system.
The user interface shows the user profile, allowing access to personal data and providing the opportunity to change one's nickname; ranking (ie, position in terms of game scores; designed to help increase motivation via a sense of competition); and a start button to allow users to begin playing. As an incentive, each time a correct answer was given in any of the app's modalities, points were added to the user's score. Each user's scores, as well as the number of modules completed, were recorded via the system.
Knowledge and mood state were assessed for both the intervention and control groups before and after the 2-week experimental period. Motivation, satisfaction, and English-language competence—the latter only measured in the intervention group—were assessed after this period.
Theoretical knowledge was evaluated at baseline (ie, the beginning of the 2-week study period) and at the end of the study period using 20 multiple-choice questions [
Motivation was assessed using the Instructional Materials Motivation Survey (IMMS) [
Mood is a short-term state of feeling; it may fluctuate within minutes or over days. Unlike emotions, moods are more transient, often unrelated to external events, and are of varying intensity [
Subjects' satisfaction with the learning methods to which they were exposed was evaluated via a questionnaire based on that reported by Brewer et al [
The subjective improvement in English-language competence (ie, a general feeling of improvement and of improvements in written comprehension, written expression, vocabulary, and global perception) was measured at the end of the study via an ad hoc questionnaire, scoring answers on an increasing 5-point Likert scale, ranging from 1 to 5. Higher values represent better outcomes. To determine the subjects' prior English-language competence, they were asked about any official English certificates they possessed before starting the study.
The normal distribution of data was verified using the Kolmogorov-Smirnov test. Results are presented as mean (SD) values. The Student
Differences between the groups in terms of the preintervention-to-postintervention change in knowledge and mood state were analyzed by repeated-measures analysis of covariance (ANCOVA), adjusting for the effects of those variables showing significant differences between groups at baseline (ie, the covariate
The study was performed in accordance with the ethical standards of the appropriate national and institutional research committees, and adhering to the Declaration of Helsinki [
A total of 99 students were enrolled, with 49 (49%) in the intervention group and 50 (51%) in the control group. The mean age of the students was 19.76 years (SD 2.74) in the intervention group versus 20.00 years (SD 3.98) in the control group (
Sociodemographic characteristics of the study subjects at baseline: the beginning of the 2-week study period.
Characteristic | Control group (n=50) | Intervention group (n=49) | ||
Age (years), mean (SD) | 20.00 (3.98) | 19.76 (2.74) | ||
|
|
|
||
|
Male | 15 (30) | 11 (22) | |
|
Female | 35 (70) | 38 (78) | |
|
|
|
||
|
Not certified | 25 (50) | 22 (45) | |
|
A2 (elementary) | 4 (8) | 2 (4) | |
|
B1 (low intermediate) | 14 (28) | 15 (31) | |
|
B2 (high intermediate) | 5 (10) | 9 (18) | |
|
C1 (advanced) | 2 (4) | 1 (2) | |
|
|
|
||
|
iOS | 19 (38) | 12 (24) | |
|
Android operating system (OS) | 31 (62) | 37 (76) | |
Baseline knowledge of subjects (scoreb), mean (SD) | 4.38 (1.21) | 4.57 (1.22) |
aEnglish levels were determined according to the Common European Framework of Reference for Languages: Learning, Teaching, Assessment [
bScores ranged from 0 (no knowledge) to 10 (highest level of knowledge).
At the end of the experimental period, no significant difference was seen between the two groups in terms of theoretical knowledge gain (intervention group mean score 0.51 [SD 1.19] vs control group mean score 0.49 [SD 1.10];
Effect of the different learning methods on knowledge gain.
Answers at study time points | Control group (n=48), mean (SD) | Intervention group (n=46), mean (SD) | ||
|
|
|
.92 | |
|
Baselineb | 8.81 (2.43) | 9.30 (2.37) |
|
|
Postintervention | 9.79 (2.48) | 10.33 (2.61) |
|
|
Difference | 0.98 (2.20) | 1.02 (2.37) |
|
|
|
|
.92 | |
|
Baseline | 11.19 (2.43) | 10.70 (2.38) |
|
|
Postintervention | 10.21 (2.48) | 9.67 (2.61) |
|
|
Difference | –0.98 (2.20) | –1.02 (2.37) |
|
|
|
|
.92 | |
|
Baseline | 4.41 (1.21) | 4.65 (1.19) |
|
|
Postintervention | 4.90 (1.24) | 5.16 (1.30) |
|
|
Difference | 0.49 (1.10) | 0.51 (1.19) |
|
aRepeated-measures analysis of covariance (ANCOVA) was used to examine the differences between groups.
b
Learning motivation in the intervention and control groups.
Learning motivation variablea,b | Control group (n=48), mean (SD) | Intervention group (n=46), mean (SD) | |
Attention (12-60) | 26.06 (9.64) | 47.59 (7.20) | <.001 |
Relevance (9-45) | 24.06 (6.58) | 34.17 (5.08) | <.001 |
Confidence (9-45) | 22.04 (6.96) | 31.83 (5.13) | <.001 |
Satisfaction (6-30) | 12.77 (4.55) | 22.44 (4.25) | <.001 |
Total IMMSa score (36-180) | 84.94 (25.37) | 136.02 (19.25) | <.001 |
aLearning motivation was measured using the Instructional Materials Motivation Survey (IMMS).
bEach category is presented with the possible range of its score in parentheses.
cThe Student
Effect of the different learning methods on the Profile of Mood States (POMS) total and subscale scores.
Mood state | Control group (n=48), mean (SD) | Intervention group (n=46), mean (SD) | |||||
|
|
|
.16 | ||||
|
Baselineb | 45.58 (8.43) | 46.78 (9.70) |
|
|||
|
Postintervention | 46.15 (9.45) | 45.00 (10.09) |
|
|||
|
Difference | 0.56 (8.88) | –1.78 (7.21) |
|
|||
|
|
|
.32 | ||||
|
Baseline | 47.38 (5.55) | 47.39 (6.82) |
|
|||
|
Postintervention | 49.21 (7.64) | 47.93 (8.64) |
|
|||
|
Difference | 1.83 (6.55) | 0.54 (6.04) |
|
|||
|
|
|
.05 | ||||
|
Baseline | 51.77 (9.52) | 52.67 (8.95) |
|
|||
|
Postintervention | 53.98 (11.79) | 51.61 (10.24) |
|
|||
|
Difference | 2.21 (7.97) | –1.07 (8.45) |
|
|||
|
|
|
.67 | ||||
|
Baseline | 54.25 (7.30) | 57.43 (6.05) |
|
|||
|
Postintervention | 51.96 (7.55) | 54.67 (6.77) |
|
|||
|
Difference | –2.29 (5.80) | –2.76 (4.82) |
|
|||
|
|
|
.63 | ||||
|
Baseline | 47.75 (7.02) | 46.59 (6.79) |
|
|||
|
Postintervention | 48.73 (9.49) | 46.80 (7.45) |
|
|||
|
Difference | 0.98 (9.24) | 0.22 (5.51) |
|
|||
|
|
|
.01 | ||||
|
Baseline | 38.54 (7.38 ) | 39.65 (8.02) |
|
|||
|
Postintervention | 42.21 (8.43) | 39.15 (7.64) |
|
|||
|
Difference | 3.67 (8.66) | –0.50 (6.62) |
|
|||
|
|
|
.09 | ||||
|
Baseline | –17677.08 (2899.91) | –17565.22 (3453.13) |
|
|||
|
Postintervention | –18831.25 (4052.59) | –17582.61 (4088.16) |
|
|||
|
Difference | –1154.17 (3652.10) | –17.39 (2767.05) |
|
aRepeated-measures analysis of covariance (ANCOVA) was used to examine the differences between groups. Significance was set at
b
Satisfaction among the intervention and control arms.
The intervention group subjects used the app an average of 3.85 (SD 1.51) days per week. The
The blended learning method led to significant improvements in motivation, mood state, and satisfaction compared to traditional teaching. Moreover, frequent users of the app showed stronger motivation and perceived greater gains in their English-language competence than did infrequent users. However, the results suggest the intervention strategy provided no benefit in terms of knowledge absorbed. The latter finding is consistent with the results of two previous randomized controlled studies evaluating online learning methods [
In this work, motivation was measured by the validated IMMS tool. Higher scores were recorded for all motivation dimensions in the intervention group. Keller defines motivation as an innate characteristic of students, but also indicates that it can be influenced by external factors such as the instructional method used [
The intervention group subjects appeared to be more involved in the learning process; they returned a significant change in two of the six mood state components assessed, which was reflected in the total score. Moreover, anger and hostility as well as confusion and bewilderment improved in the intervention group subjects, while these became worse (ie, increased scores) among those in the control group. The use of the app might thus have helped improve negative mood states. Pekrun et al [
Finally, satisfaction was greater among the intervention group subjects. It seems clear that the role of m-learning was perceived as an appropriate complement to traditional lecturing. With the development of advanced technologies, smartphones can be an effective learning tool for students. The possibility of reviewing classes whenever and wherever one likes, with tailored feedback (ie, scores for the games and quizzes), may have promoted the transfer of knowledge into their short-term and even long-term memories [
This study suffers from the limitation that it lasted only 2 weeks. This short time frame was due to external factors such as a funder-imposed deadline. Further, the iPOT app was created for use as a post-teaching period review tool, not as a study tool in itself. No difference was detected between the amount of knowledge absorbed by the members of the two groups. However, if, as the results show, the use of the app motivates students, a significant difference in what is learned might be expected after a longer period of use. Further work should investigate this.
Students who received instruction via blended learning involving m-learning showed greater motivation, a better mood state, and greater satisfaction than those who received traditional lectures alone, although no difference was seen in terms of the amount of knowledge absorbed. Longer studies are needed to determine whether the improvements in these factors persist and whether they eventually translate into more knowledge being absorbed.
iPOT video tutorial.
CONSORT-EHEALTH checklist (V 1.6.1).
analysis of covariance
intraclass correlation coefficient
Instructional Materials Motivation Survey
mobile augmented reality
mobile learning
operating system
Profile of Mood States
Quick Response
Secure Sockets Layer
Total Mood Disturbance
Unit of Excellence on Exercise and Health
We are grateful to Adrian Burton for assistance with the English language. The authors are also grateful to the students who participated in the study. This work was funded by a grant from the Educational Innovation Unit of the University of Granada, Spain (PID 16-54). Additional funding was provided by the University of Granada, Plan Propio de Investigación 2016, Excellence actions: Units of Excellence; Unit of Excellence on Exercise and Health (UCEES).
None declared.