This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
E-learning and blended learning approaches gain more and more popularity in emergency medicine curricula. So far, little data is available on the impact of such approaches on procedural learning and skill acquisition and their comparison with traditional approaches.
This study investigated the impact of a blended learning approach, including Web-based virtual patients (VPs) and standard pediatric basic life support (PBLS) training, on procedural knowledge, objective performance, and self-assessment.
A total of 57 medical students were randomly assigned to an intervention group (n=30) and a control group (n=27). Both groups received paper handouts in preparation of simulation-based PBLS training. The intervention group additionally completed two Web-based VPs with embedded video clips. Measurements were taken at randomization (t0), after the preparation period (t1), and after hands-on training (t2). Clinical decision-making skills and procedural knowledge were assessed at t0 and t1. PBLS performance was scored regarding adherence to the correct algorithm, conformance to temporal demands, and the quality of procedural steps at t1 and t2. Participants’ self-assessments were recorded in all three measurements.
Procedural knowledge of the intervention group was significantly superior to that of the control group at t1. At t2, the intervention group showed significantly better adherence to the algorithm and temporal demands, and better procedural quality of PBLS in objective measures than did the control group. These aspects differed between the groups even at t1 (after VPs, prior to practical training). Self-assessments differed significantly only at t1 in favor of the intervention group.
Training with VPs combined with hands-on training improves PBLS performance as judged by objective measures.
Basic life support training, such as for pediatric basic life support (PBLS), is usually simulation-based with the need for evaluating learners’ performances [
For promoting clinical reasoning and decision making, virtual patients (VPs) are known for being effective [
In this study, we investigated the effect of VPs combined with standard simulation-based PBLS training on the acquisition of clinical decision-making skills and procedural knowledge, objective skill performance, and self-assessment. Our hypotheses were that preparation with VPs would yield (1) superior clinical decision making and procedural knowledge, (2) an objectively better performance of PBLS after the training, and (3) better self-assessment after working with VPs and after exposure to standard training.
We used a two-group randomized trial design (see
Study design.
All instruments were pilot-tested on video recordings of PBLS demonstrations by student tutors and faculty before implementation, and revisions were made to ensure clarity and content validity. We particularly tested the estimated and calculated temporal scores adapted from international recommendations [
Participants were asked about their age, sex, and level of qualification in emergency medicine. For subgroup analysis we identified participants who were qualified as paramedics or had some similar training—qualifications that include PBLS training.
We developed a key-feature test according to published guidelines [
Two raters scored the performed algorithm for its correct order. Each step of the sequence was given 2 points if it was done in the correct algorithmic order. It was given 1 point if it had been performed in an incorrect algorithmic order. No points were assigned if the step had not been undertaken at all (see
Concrete temporal recommendations for three procedural steps of the PBLS algorithm are as follows [
1. Every rescue breath should take 1.0 to 1.5 s for inspiration plus time for expiration.
2. Assessment of the signs of life and circulation may not take longer than 10 s.
3. Chest compressions should be given at a frequency of at least 100/min, not exceeding 120/min.
With these recommendations being followed, the optimal temporal specifications for the initial five rescue breaths, the circulation check, and the four cardiopulmonary resuscitation (CPR) cycles were estimated and calculated (see
Two group-blinded video raters with expertise in PBLS scored the procedural quality of the participants’ PBLS skills. The scores were averaged for further analysis. We used a scoring form in trichotomous fashion, with 2 points for correct performance, 1 point for minor deficits, and no points for major deficits (see
We developed a self-assessment instrument consisting of seven items on procedural knowledge and seven items on procedural skills (see
For individual preparation of the training, we developed and distributed to both groups a paper handout on PBLS. Such handouts are commonly used as preparation for undergraduate skills laboratories [
Participants were trained in a single-rescuer scenario: from finding an unresponsive child, to the emergency call after 1 min (about four cycles) of CPR according to current guidelines [
Screenshot of CAMPUS-Software showing a virtual patient.
The Ethics Committee of the Medical Faculty Heidelberg granted ethical approval for this study (EK No. S-282/2014). All collected data were pseudonymized. We affirmed with participants by written informed consent that their participation was voluntary, that they could not be identified from the collected data, and that no plausible harm could arise from participation in the study.
We offered participation in this study to a total of about 480 third- and fourth-year medical students at Heidelberg Medical School by group emails and bulletin boards. Invited students had already completed basic life support (BLS) training but had had no PBLS training yet. Announcements were worded as invitations to a special PBLS course and educational study without mentioning e-learning in particular. At an orientation meeting, prospective students enrolled themselves onto a numbered list, unaware of group allocation, which was randomly distributed by numbers.
We selected and trained two raters to score videotaped performances with the help of best-practice videos of senior faculty. Rater training included reviewing the case content and objectives, and an introduction to the rating schemes. Videotaped examples with different levels of procedural quality were discussed for calibration of the intended use of the schemes. We chose a senior pediatric consultant and a pediatric intensive care nurse practitioner, each of whom was an experienced facilitator for pediatric emergency simulations. Raters were blinded to group classification of all video records.
Results are presented as the mean ± standard deviation per group and given as the percent of the maximum achievable scores. Data were checked for normal distribution using the Kolmogorov-Smirnov test. If a presumed normal distribution was accepted, statistical differences were evaluated using the unpaired
Scoring results are depicted in
Key-feature test, performance, and self-assessment scores.
Scored items | Scores, mean (SD) or n (%) |
|
||
|
Control group (CG) | Intervention group (IG) |
|
|
|
|
|
|
|
|
t0 a | 31.0 (12.9) | 34.8 (17.1) | .34 |
|
t1 b | 68.8 (16.3) | 92.2 (4.7) | < |
|
|
|
< |
|
|
|
|
|
|
|
t1 | 72.0 (17.7) | 93.4 (7.1) | < |
|
t2 d | 95.7 (7.2) | 99.8 (1.1) |
|
|
|
< |
< |
|
|
|
|
|
|
|
t1 | 43.3 (23.4) | 67.6 (21.4) | < |
|
t2 | 55.8 (27.8) | 82.4 (17.8) | < |
|
|
|
|
|
|
|
|||
t1 | 107.7 (35.2) | 88.1 (12.6) |
|
|
|
t2 | 95.2 (16.2) | 78.1 (10.2) | < |
|
|
.05 | < |
|
|
|
|
|
|
|
t1 | 48.8 (20.2) | 68.2 (15.0) | < |
|
t2 | 84.4 (11.2) | 89.4 (9.2) | .07 |
|
|
< |
< |
|
|
|
|
|
|
|
t1 | 0/30 (0) | 5/27 (19) |
|
|
t2 | 17/30 (57) | 23/27 (85) |
|
|
|
|
|
|
|
t0 | 29.1 (16.3) | 27.2 (18.8) | .68 |
|
t1 | 59.6 (15.8) | 72.3 (11.7) |
|
|
t2 | 87.4 (8.6) | 88.4 (7.8) | .63 |
aPrepreparation assessment (t0).
bPostpreparation assessment (t1).
cItalicized
dPosttraining assessment (t2).
ePediatric basic life support (PBLS).
Key-feature test, performance, and self-assessment scores. Scores are given as the percent of the maximum achievable scores (*
A total of 57 participants completed the training and all surveys were included in this study—30 (53%) in the control group (CG) and 27 (47%) in the intervention group; approximately 11.9% (57/480) of all eligible students. Out of 60 initial participants, 3 (5%) were excluded due to nonappearance at the training session; all participants of the intervention group processed the VPs completely as requested. Participants’ mean age was 24.2 years (SD 2.6) (16/30, 53% female) in the control group and 24.1 years (SD 3.1) (17/27, 63% female) in the intervention group. Of the 57 participants, there were 5 out of 30 (17%) PBLS-qualified participants (paramedics) in the control group and 4 out of 27 (15%) in the intervention group.
There was no significant difference in the key-feature test results between the control group and intervention group at t0 (31.0%, SD 12.9 vs 34.8%, SD 17.1;
Regarding adherence to the algorithm, the intervention group was already better than the control group at t1 (93.4%, SD 7.1 vs 72.0%, SD 17.7;
The intervention group already showed significantly better adherence to temporal specifications than the control group at t1 (67.6%, SD 21.4 vs 43.3%, SD 23.4;
The interrater reliability coefficient was .71 indicating a sufficient level of interrater agreement [
The performance quality score of the intervention group was significantly superior to that of the control group at t1 (68.2%, SD 15.0 vs 48.8%, SD 20.2;
The global ratings of competence showed significant differences in favor of the intervention group, again already at t1 and continuing at t2 (0/30 CG vs 5/27 IG rated “competent”,
There was no significant difference in the self-assessment means of the two groups at t0 (29.1%, SD 16.3 CG vs 27.2%, SD 18.8 IG;
After identifying and excluding PBLS-qualified participants, there were no changes in statistical significances in any of the calculations. The level of significance did not differ by the power of 10.
In this randomized controlled trial, we investigated the impact of an additional preparation with VPs on the improvement of objective and subjective learning outcomes of skill acquisition when combined with standard PBLS training. The control group and intervention group were comparable in terms of their self-assessment and procedural knowledge during the prepreparation assessment. However, after addition of practical training, the intervention group demonstrated significantly better performance in key aspects of PBLS than did the control group, although self-assessment ratings were similar. Also, after practicing with VPs, the intervention group had already demonstrated superior skills, even before the hands-on training in terms of objective skill performance, procedural knowledge, and self-assessment.
After using VPs as an interactive preparation, the intervention group showed significantly improved clinical decision-making skills and procedural knowledge. Also, their PBLS skill performance was superior to that of the control group after the preparation period in regard to objective performance measures, including adherence to the algorithm, temporal demands, and procedural quality. This is in line with existing reports that such electronic learning activities improve both knowledge and skills [
We assume that VPs facilitated application of acquired clinical decision-making skills and procedural knowledge. Interactivity and feedback in VPs, which included interactive graphics and video clips, might have enhanced the learning process beyond the use of media, as in other approaches. It is well known that educational feedback, such as that given in the VPs, is the most important feature of simulation-based education [
The presented results support the subjective perceptions of students and tutors [
In their self-assessments, the participants of the intervention group judged themselves superior to those in the control group after the preparation period. Objective findings in their scored performances support these ratings. After their practical training, however, the self-assessments of the two groups were similar. In contrast, the intervention group still had superior objective scores regarding skill performance. Self-assessments are not necessarily correlated with performance; for example, postgraduate practitioners have limited ability to self-assess accurately, as shown by Davis et al [
The assessments of clinical decision-making skills and procedural knowledge, practical performance, and self-assessment combine relevant and detailed objective and subjective measures for elucidating the learning effects of this approach. This is one of the first studies that provides objective data that support how effectively VPs can foster the acquisition of PBLS skills.
Participants’ VP case completions were monitored to validate their workup, but the validation was not done in a controlled environment that allowed evaluation of participants’ efforts. Accordingly, efforts on the workup of handouts were also not assessed. Workup of VPs might have led to more motivation for learning even though the study was not announced as an e-learning study attracting mainly tech-savvy students. Because both groups had significantly increased procedural knowledge after the preparation period, we assumed that motivation for preparation might not have been very different. Result details, for example, of the temporal scoring, suggest that VPs address skills not affected by paper-based learning materials. However, group differences might also have been influenced by different durations of their efforts to learn in addition to different modalities. Additionally, both groups were not limited in their access to other learning resources than those provided. Also, most of the instruments used in this study have not been validated formally, although all were developed based on current literature and were pilot-tested. Finally, the sample size is rather limited, thereby providing limited power to investigate differences between groups.
The blended learning approach described herein leads to improved outcomes of practical skill acquisition compared with a standard approach. Even before having practical training, preparation with VPs leads to improved practical performances as well as better clinical decision-making skills and procedural knowledge. Further studies are necessary to understand the specific benefit of using VPs regarding clinical skill acquisition and its sustainability.
Key-feature test.
Adherence to algorithm scoring form.
Temporal measures scoring form.
Performance quality scoring form.
Self-assessment instrument.
basic life support
control group
cardiopulmonary resuscitation
case 2 intraclass correlation coefficient
intervention group
pediatric basic life support
prepreparation assessment
postpreparation assessment
posttraining assessment
virtual patient
This study was funded by general departmental funds without involving third party funds.
None declared.