Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 12.01.15 in Vol 17, No 1 (2015): January

This paper is in the following e-collection/theme issue:

    Original Paper

    Designing and Evaluating an Interactive Multimedia Web-Based Simulation for Developing Nurses’ Competencies in Acute Nursing Care: Randomized Controlled Trial

    1National University of Singapore, Singapore, Singapore

    2The University of Newcastle, Newcastle, Australia

    3National University Hospital, Singapore, Singapore

    Corresponding Author:

    Sok Ying Liaw, PhD

    National University of Singapore

    Level 2, Clinical Research Centre, Block MD 11

    10 Medical Drive

    Singapore, 117597

    Singapore

    Phone: 65 65167451

    Fax:65 67767135

    Email: nurliaw@nus.edu.sg


    Related Article:

    This is a corrected version. See correction statement: http://www.jmir.org/2015/3/e75/

    ABSTRACT

    Background: Web-based learning is becoming an increasingly important instructional tool in nursing education. Multimedia advancements offer the potential for creating authentic nursing activities for developing nursing competency in clinical practice.

    Objective: This study aims to describe the design, development, and evaluation of an interactive multimedia Web-based simulation for developing nurses’ competencies in acute nursing care.

    Methods: Authentic nursing activities were developed in a Web-based simulation using a variety of instructional strategies including animation video, multimedia instructional material, virtual patients, and online quizzes. A randomized controlled study was conducted on 67 registered nurses who were recruited from the general ward units of an acute care tertiary hospital. Following a baseline evaluation of all participants’ clinical performance in a simulated clinical setting, the experimental group received 3 hours of Web-based simulation and completed a survey to evaluate their perceptions of the program. All participants were re-tested for their clinical performances using a validated tool.

    Results: The clinical performance posttest scores of the experimental group improved significantly (P<.001) from the pretest scores after the Web-based simulation. In addition, compared to the control group, the experimental group had significantly higher clinical performance posttest scores (P<.001) after controlling the pretest scores. The participants from the experimental group were satisfied with their learning experience and gave positive ratings for the quality of the Web-based simulation. Themes emerging from the comments about the most valuable aspects of the Web-based simulation include relevance to practice, instructional strategies, and fostering problem solving.

    Conclusions: Engaging in authentic nursing activities using interactive multimedia Web-based simulation can enhance nurses’ competencies in acute care. Web-based simulations provide a promising educational tool in institutions where large groups of nurses need to be trained in acute nursing care and accessibility to repetitive training is essential for achieving long-term retention of clinical competency.

    J Med Internet Res 2015;17(1):e5

    doi:10.2196/jmir.3853

    KEYWORDS



    Introduction

    Acquiring nursing competencies in assessing and managing acutely ill patients has been identified as a crucial learning goal in the acute care hospital to reduce the occurrence of adverse events such as cardiopulmonary arrest [1,2]. Previous studies have shown the benefits of using mannequin-based simulation in acute nursing care training [3,4]. As simulation places the learners in a realistic situation, it is considered an authentic learning activity to prepare learners for their real-life work. Lebow described authentic activity as “experiences of personal relevance that permit learners to practice skills in environments similar to those in which the skills will be used” [5]. The authentic learning activities embedded in simulation enable the learners to develop knowledge, skills, and attitudes in an integrative whole, which facilitates the transfer to future real-life work settings [6]. However, developing and implementing an authentic learning environment using simulation is often limited by the availability of simulation facilities, the requirement for trained simulation instructors, and the small numbers of learners involved in the training at one time [7]. Given the resource-intensive nature of simulation, it would be a challenge for institutions that need to train large groups of nurses. Nonetheless, such limitations could be eased with the development of Web-based learning (WBL) technology.

    With technological advancements, WBL can provide an authentic learning context that responds to nurses’ needs and experiences in clinical practice. WBL is endorsed as an essential educational tool for lifelong learning and professional development [8]. Previous studies did not show any differences between WBL and face-to-face learning in the acquisition of knowledge, skills, and satisfaction [9]. Additionally, nurses perceived that WBL was suited to meeting their working conditions and needs including conflicting work schedules and intensive workload [10].

    Advances in multimedia technology make the design of authentic learning activities increasingly feasible in WBL. Multimedia refers to technology for presenting visual and verbal material [11]. Instructional designers can use media such as text, graphics, audio, and video to simulate real-world environments where learners can perform authentic tasks. In addition, multimedia technology makes it possible for situating simulations in WBL (known as Web-based simulation). One approach is for learners to play roles in goal-based scenarios driven by assessment [12]. Web-based simulations using virtual patients are now popular in health care education for simulating authentic clinical experiences [13].

    A review of literature between 2000 and 2014 identified 18 Web-based simulation programs in nursing education with most involving teaching an aspect of procedural patient care and others on interpersonal communication skills or technique skills for equipment use. Descriptive studies reported high levels of acceptance of Web-based simulations by nursing students [14]. Using experimental studies, some of the Web-based simulations were tested by comparing them with mannequin-based simulation. These studies revealed that Web-based simulations are at least as good as, if not better than, mannequin-based simulations in teaching the necessary skills and knowledge to develop competent nursing students [15,16]. As the existing Web-based simulations have a predominant focus on pre-registered health professional education [14], further study is recommended to explore the use of Web-based simulation in continuing health professional education [17].

    It appears Web-based simulation is more cost-effective than mannequin-based simulation for training large groups of hospital nurses. The successful implementation of mannequin-based simulation for undergraduate nurses in acute nursing care has prompted us to develop an alternative authentic learning strategy in continuing nursing education of hospital nurses [18]. The use of WBL is still fairly new in the field of acute nursing education. WBL instructional design methods vary considerably, and research is needed to find the most effective method [19]. Thus, the aim of this study is to describe the design, development, and evaluation of an interactive multimedia Web-based simulation for developing nurses’ competencies in acute nursing care.


    Methods

    Overview

    The instructional design model developed by Smith and Ragan [20] was used to guide the design, development, and evaluation of the interactive multimedia Web-based simulation. The model focuses on three key activities: analysis, strategy, and evaluation.

    Analysis

    Learners

    The targeted learners were registered nurses (RNs) who have acquired either a diploma or baccalaureate nursing degree. In caring for acutely ill patients, RNs play an important role in recognizing patient deterioration, conveying their assessments to health care staff, and providing timely and appropriate intervention before the arrival of appropriate help.

    Learning Objectives

    With the analysis of nurses’ roles in caring for acutely ill patients, three specific learning objectives were identified: (1) understanding the underlying physiological signs of patient deterioration, (2) recognizing and managing deteriorating patients, and (3) communicating effectively about patient deterioration.

    Learning Tasks

    An important learning task to help nurses recognize the early signs of patient deterioration involves understanding the physiological compensatory mechanism and pathophysiology underpinning changes in vital signs. Here, the “ABCDE” (Airway, Breathing, Circulation, Disability and Expose/Examine) and “ISBAR” (Identity, Situation, Background, Assessment and Recommendation) mnemonics provide frameworks to guide nurses in assessing, managing, and reporting on patients’ deterioration. Specifically, the ABCDE mnemonic encourages a systematic approach to recognizing and responding to deteriorating ward patients. The ISBAR mnemonic provides a structured communication tool for nurses to report on patients’ deterioration. A panel of national experts developed an evaluation tool that assessed the suitability of tasks to be performed by ward nurses in response to patient deterioration. The learning tasks were adapted from this evaluation tool. An international panel of nursing and medical experts further validated the evaluation tool [21].

    Learning Context

    An educational program using simulation was implemented in a Singapore acute care hospital to support early recognition of and intervention in patient deterioration. Despite the benefits of simulation training, there are logistical and resource issues in implementing this kind of training for a large group of nurses. Nurse educators and hospital administrators highly recommended an alternative effective learning strategy that can overcome these difficulties and be accessible to all the hospital nurses.

    Strategy

    Overview

    The learning objectives guided the development of the learning content. As shown in Figure 1, the content delivery sequence includes four key events: (1) stimulate motivation, (2) acquisition of knowledge, (3) practice and feedback, and (4) formative assessment (see Multimedia Appendix 1 for a video). These are facilitated through a variety of instructional activities including animation video, multimedia instructional material, virtual patient simulation, and online quizzes.

    Figure 1. Instructional activities in Web-based simulation.
    View this figure
    Stimulate Motivation

    To arouse the learners’ motivation to learn, a narrative about a patient experiencing an adverse event (cardiopulmonary arrest) is presented through an animation video. The narrative highlights that such an event can be prevented through the early recognition of patient deterioration (Figure 1a).

    Acquisition of Knowledge

    Information about the underlying physiological signs of patient deterioration is presented in an animation showing two nurses in conversation (Figure 1b). Onscreen text and illustrations are used to explain the tasks involved in assessing, managing, and reporting on the acutely ill patient. These performance steps are organized using evidence-based mnemonic strategies (Figure 1c). The ABCDE mnemonic is employed to help organize the steps of assessing and managing the deteriorating patient, while the ISBAR mnemonic is used to show the steps for reporting on the deteriorating patient. Rationales from evidence-based practice on these steps are elaborated to help learners understand why these steps are important as well as how to understand them. Illustrations and audio lung sounds are employed to assist users in understanding the facts (Figure 1d).

    Practice and Feedback

    Virtual patient simulation is designed to give learners the opportunity to gain practical experience related to the information being learned. Common deteriorating conditions (ie, airway obstruction, breathlessness, hypotension, tachycardia, oliguria, altered consciousness, and abnormal temperatures) associated with acute medical conditions are embedded in five simulation scenarios pertaining to events occurring on different days of the virtual patient’s admission (Figure 1e). In each scenario, the learner emulates the role of a nurse assessing and managing the deteriorating virtual patient by selecting actions from the ABCDE control menus. Immediate feedback, including physiological changes, is programmed into the system to respond to the learner’s actions. ISBAR control menus are used to aid learners in reporting on the virtual patient’s deterioration (Figure 1f). At the end of each scenario, the learners are asked debriefing questions to help them reflect on their experiences. In addition, using an evaluation tool, the learners receive feedback on the appropriate and inappropriate actions taken in the simulation scenario (Figure 1g).

    Formative Assessment

    Formative assessment is seamlessly integrated with the simulation activity. The learners’ performances in each scenario are scored using a validated evaluation tool. Multiple-choice questions related to each simulation scenario are constructed to assess learners’ knowledge of the subject content (Figure 1h).

    Evaluation

    Design and Sample

    A prospective, randomized controlled trial (RCT) with a pretest-posttest design was conducted from November to December 2013. The National University of Singapore Institutional Human Research Ethics Board approved the study. A total of 70 registered nurses, with less than 5 years of work experience and who were working in general ward units, were recruited from an acute care tertiary hospital in Singapore. A sample of 60 participants (30 from each group) was considered adequate to achieve statistical power at the 5% level of significance [20]. Allowing for an attrition rate of 10%, a total of 35 participants were recruited to each arm. They were randomly assigned to an experiment group (N=35) and a control group (N=35) using a computerized random number generator. However, three participants from the control group did not turn up for the study, leaving that group with 32 participants. All participants were given a participant information sheet that explained the purpose of the study.

    Procedure

    Figure 2 depicts the flow of the study procedure. All participants completed a questionnaire on demographic information. The participants undertook a performance pretest consisting of a simulation-based assessment that took place at the university simulation laboratory. They hid their identities from the raters by putting on caps, gowns, and masks. Following an orientation to the simulation set-up, each participant was given a test scenario with the patient simulator displaying signs and symptoms of clinical deterioration. Each participant was given 15 minutes to assess and manage the deteriorating patient simulator. The entire simulation process was recorded on video. Immediately after the performance pretest, the participants in the experimental group were brought individually into a room with a computer set-up to undertake a 3-hour Web-based simulation. After completing the learning, the participants were asked to complete a questionnaire to evaluate their perception of the Web-based simulation. About a week after the pretests and intervention, the participants from both groups were scheduled to undertake the performance posttests on simulation-based assessment individually, which were similar to the performance pretests.

    Figure 2. Flow of data collection.
    View this figure
    Data Collection and Instruments

    Research staff observed and rated the recorded performance using a validated tool known as the RAPIDS (Rescuing a Patient in Deteriorating Situations) tool. The psychometric properties of the RAPIDS tool, including content and construct validity, and interrater reliability were tested and supported in a previous study [21]. Of the recorded videos, 30% (20/67) were randomly selected in this study for rerating by another member of the research staff. There was excellent interrater reliability between the 2 raters, with a high intraclass correlation coefficient (ICC) of .98 (95% CI 0.95-0.99).

    An 18-item scale with four subscales (system quality, information quality, user satisfaction, and net benefit) was administered to explore the participants’ perception of the Web-based simulation. This scale was adapted and modified from the e-learning systems success (ELSS) scale [22]. The study obtained a high internal consistency of this scale with Cronbach alpha at .97. Two questions were added to allow for open-ended comments.

    Data Analysis

    Descriptive statistics were computed for the demographic variable and participants’ perceptions of the Web-based simulation. Differences in demographic characteristics between the two groups were examined by a chi-square test and t test. Interrater reliability was assessed using ICC. Changes between baseline and posttest scores were determined by a paired t test. Analysis of covariance (ANCOVA) was performed to evaluate the posttest scores between groups with baseline scores as a covariate. The comments for the two open-ended questions were coded and analyzed for recurring themes.


    Results

    Most of the participants were female (97%) and Chinese (62.7%), with an average age of 25.58 years (SD 3.19). Demographic characteristics including gender (P=1.00), ethnicity (P=.98), age (P=.12), years of work experience (P=.10), and highest qualification (P=.40) were similar for both groups of participants. This supported the randomization and homogeneity of the participants between the groups.

    Figure 3 shows that the clinical performance pretest scores did not differ significantly between the experimental and control groups. Within the groups, the clinical performance posttest scores for the experimental group were significantly higher than the pretest scores (P<.001). No significant difference was found between the clinical performance pretest and posttest scores for the control group. Between-group comparisons found significantly higher clinical performance posttest scores (P<.001) for the experimental group than the control group after controlling the pretest scores.

    The mean scores from the participants’ rating (experimental group) on the 7-point scale indicated that that they were highly satisfied with the Web-based simulation (mean 6.00, SD 0.79), positive about the quality of the system (mean 6.04, SD 0.68) and information (mean 5.13, SD 0.62), and thought highly of the net benefits (mean 6.11, SD 0.71) of the program.

    Three themes with categories emerged from the written comments on the most valuable aspects of the Web-based simulation: (1) relevance to practice as it provides useful information (ie, “provide specific information which is very useful for us nurses”) and allows application in a practice setting (ie, “can apply in real-life situation”), (2) instructional strategies including animation, e-simulation, and mnemonics (ie, “ABCDE and ISBAR” make it simple and clear to understand”) fostering problem solving through knowledge gained (ie, “understand the signs and symptoms of early deterioration”), and (3) the use of critical thinking (ie, “utilizes critical thinking to choose appropriate nursing assessment and intervention”). Three main themes emerged on ways to improve the WBL program: more variation, more information, and technical features.

    Figure 3. Pretests and posttests scores on clinical performance.
    View this figure

    Discussion

    Principal Findings

    This randomized controlled study provided evidence of the effectiveness of a Web-based simulation in improving hospital nurses’ acute care competencies. The development of these competencies required the hospital nurses to apply and integrate a broad range of knowledge, skills, and attitudes. In this study, we developed a Web-based simulation that provided opportunities for the learners to work on authentic nursing activities in an authentic learning environment. Authentic activities are tasks learners perform in solving real-life problems [23]. An authentic environment resembling a hospital ward was developed to enhance the learning experience. By providing a context that reflects the way knowledge and skills will be used in actual life, the authentic learning environment stimulates the learners to develop competencies relevant to their working lives [24].

    Apart from real-world relevance, a variety of instructional strategies that were drawn from instructional design models were incorporated into the Web-based simulation to develop the nurses’ clinical competencies. The nurses valued the instructional strategies as well as the problem-solving skills fostered through the learning process. The learning process applied theories from a range of fields including cognitive overload, motivation, and multimedia learning, The process began by stimulating the learners’ motivation followed by the acquisition of factual knowledge of the learning tasks using multimedia such as animation video, texts, audio, and illustrations. The importance of incorporating a variety of activities to gain and sustain learner attention was identified as a key element in a motivation model of instructional design [25]. The acquisition of factual knowledge was supported by the use of mnemonics (ABCDE and ISBAR) to assist the learners with mental models and cognitive aids [26].

    Drawing on the experiential learning theory [27], the virtual patients in the Web-based simulation provided opportunities for the learners to apply their knowledge through practice in multiple scenarios. Each scenario involved the application of knowledge to problem solve the deteriorating virtual patient by collecting and integrating patient assessment data to arrive at a nursing diagnosis and provide nursing interventions. The use of virtual patients in professional health education has been attributed to the development of clinical reasoning [28-30]. The outcome of this study sheds light on the effectiveness of applying several instructional design strategies based on tested theories in the acquisition of clinical competency. This calls on educators to apply these strategies in the design and implementation of e-learning.

    Our findings are consistent with several previous studies that demonstrated the effectiveness of WBL in improving learning outcomes when compared with no intervention [9,31]. Although evidence from previous studies did not indicate the superiority of WBL over other non-computer training methods, it has been shown to be at least as good as, if not better than, instructor-led methods [32] including mannequin-based simulation [15]. Our study demonstrated that, with advances in multimedia, the features of mannequin-based simulation can be built into the Web-based simulation through the use of virtual patients. A systematic review identified that the use of instructional design features including interactivity, practice exercises, repetition, and feedback can favorably influence learning outcomes [32]. These features, which are also used in mannequin-based simulation, were employed in the design of the virtual patients. Learners had the opportunity to engage with the virtual patient in multiple and varied scenarios to enable them to gain repetitive practice. The ability to give various types of feedback, including verification (eg, verifying the learners’ actions) and elaborated feedback (eg, elaborate explanation-style feedback), was incorporated into the design of virtual patients. “Deliberate practice with multiple examples and feedback” has been highlighted as the best approach to the acquisition of clinical competency [33]. Hence, it is evident that the virtual patient in this Web-based simulation has a significant impact on improving the learners’ clinical competency.

    Participants’ positive evaluation of the Web-based simulation, along with improvement in their learning outcomes, is consistent with research findings relating to the efficacy of mannequin-based simulation in acute nursing care training [2,3]. However, compared with mannequin-based simulation, Web-based simulation would be a viable option in institutions where large numbers of learners have to be trained. Given the requirements for simulation facilities, facilitators, and small-group learning, mannequin-based simulation has constraints in providing scalable and sustainable training [34]. In contrast, Web-based simulation allows repetitive training, which is absolutely essential for achieving long-term retention of clinical competency [7]. Although Web-based simulation has fewer constraints than mannequin-based simulation, it should not be seen as a substitute for mannequin-based simulation. Unlike Web-based simulation, mannequin-based simulation provides “hands on” kinesthetic learning in a realistic and dynamic situation, which is considered to prepare nurses well for their real-life work. Both are different learning strategies but could be used as complementary learning tools, forming part of a blended-learning strategy to optimize clinical competency gains [7,35].

    Strengths and Limitations

    A rigorous research methodology, RCT, was used to evaluate the learning outcomes. However, the quality of the evidence could be limited by the no-intervention control group. Given that this study is looking at the development of a new WBL program for hospital nurses, the no-intervention controlled study is still considered valuable in the early stages of an innovation. The survey evaluation, particularly the qualitative data, shed light on the value of the program and ways to enhance it. Comparisons of other WBL instructional designs could be conducted in future studies to enhance the program in the future. We tested the learners’ clinical performances using a validated and tested instrument. However, due to logistic constraints, we did not measure the long-term retention of clinical performances, which can deteriorate over time. Future studies could evaluate this competence over a longer period of time. The present study may have shown the effectiveness of Web-based simulation in the context of transferring learning from the Web-based to the simulated environment. Moving forward, future studies could determine the higher-order outcomes of the program on actual clinical practice by evaluating nurse behaviors in practice and their effects on patient care.

    Conclusions

    Changes in health care delivery mean that nurse educators need to ensure the competency of all hospital nurses in acute nursing care for optimal patient care outcomes. In this study, we described how Web-based simulation can be effectively implemented to create an authentic nursing activity for developing hospital nurses’ competencies in acute nursing care. A broad array of instructional strategies based on tested theories, including animation video, multimedia instructional material, virtual patient, and online quizzes, were incorporated into the interactive multimedia Web-based simulation. Using a randomized controlled study, we demonstrated the effectiveness of the Web-based simulation in developing nurses’ clinical competencies. This study provides evidence for the acceptance of this Web-based simulation for continuing nursing education among hospital nurses. Nurse educators can use Web-based learning technology to improve the efficiency and effectiveness of educational intervention in the face of pedagogical challenges, especially those posed by mannequin-based simulation. More research is needed to inform the effective use of Web-based simulation by aligning and optimizing its use with other educational technologies, as part of a blended-learning strategy.

    Acknowledgments

    The authors would like to thank the nurses of the National University Hospital for their assistance with this study; the Centre for Healthcare Simulation for their support with this study; and the NUHS Medical Publications Support Unit, Singapore, for providing editing services for this manuscript. This study was supported by a National University of Singapore research start-up grant funded by Yong Loo Lin School of Medicine and Alice Lee Centre for Nursing Studies.

    Conflicts of Interest

    The first author is the owner and developer of the simulation software.

    Multimedia Appendix 1

    Multimedia interactive Web-based simulation.

    MP4 File (MP4 Video), 8MB

    Multimedia Appendix 2

    CONSORT-EHEALTH checklist V1.6.2 [36].

    PDF File (Adobe PDF File), 983KB

    References

    1. Fuhrmann L, Østergaard D, Lippert A, Perner A. A multi-professional full-scale simulation course in the recognition and management of deteriorating hospital patients. Resuscitation 2009 Jun;80(6):669-673. [CrossRef] [Medline]
    2. Liaw SY, Scherpbier A, Klainin-Yobas P, Rethans JJ. A review of educational strategies to improve nurses' roles in recognizing and responding to deteriorating patients. Int Nurs Rev 2011 Sep;58(3):296-303. [CrossRef] [Medline]
    3. Kinsman L, Buykx P, Cant R, Champion R, Cooper S, Endacott R, et al. The FIRST2ACT simulation program improves nursing practice in a rural Australian hospital. Aust J Rural Health 2012 Oct;20(5):270-274. [CrossRef] [Medline]
    4. Liaw SY, Rethans JJ, Scherpbier A, Piyanee KY. Rescuing A Patient In Deteriorating Situations (RAPIDS): A simulation-based educational program on recognizing, responding and reporting of physiological signs of deterioration. Resuscitation 2011 Sep;82(9):1224-1230. [CrossRef] [Medline]
    5. Lebow D. Constructivist values for instructional systems design: five principles toward a new mindset. Educational Technology Research and Development 1993;41(3):4-16.
    6. Bastiaens TJ. Design of an authentic e-learning environment. Hershey, PA: IGI Global; 2012.
    7. Bonnetain E, Boucheix JM, Hamet M, Freysz M. Benefits of computer screen-based simulation in learning cardiac arrest procedures. Med Educ 2010 Jul;44(7):716-722. [CrossRef] [Medline]
    8. Farrell GA, Cubit KA, Bobrowski CL, Salmon P. Using the WWW to teach undergraduate nurses clinical communication. Nurse Educ Today 2007 Jul;27(5):427-435. [CrossRef] [Medline]
    9. Lahti M, Hätönen H, Välimäki M. Impact of e-learning on nurses' and student nurses' knowledge, skills, and satisfaction: a systematic review and meta-analysis. Int J Nurs Stud 2014;51(1):136-149. [CrossRef]
    10. Karaman S. Nurses' perceptions of online continuing education. BMC Med Educ 2011;11:86 [FREE Full text] [CrossRef] [Medline]
    11. Mayer RE. The promise of multimedia learning. The Cambridge Handbook of Multimedia Learning. Cambridge, MA: Cambridge University Press; 2009.
    12. Holt D, Segrave S, Cybulski JL. Professional education using e-simulations: benefits of blended learning design. Hershey, PA: IGI Global; 2012.
    13. Cook DA. Agency for Healthcare Research and Quality. 2013 Mar. The literature on health care simulation education: What does it show? Perspective   URL: http://webmm.ahrq.gov/perspective.aspx?perspectiveID=138 [accessed 2014-12-04] [WebCite Cache]
    14. Cant RP, Cooper SJ. Simulation in the Internet age: The place of Web-based simulation in nursing education. An integrative review. Nurse Educ Today 2014 Dec;34(12):1435-1442. [CrossRef] [Medline]
    15. Liaw SY, Chan SW, Chen FG, Hooi SC, Siau C. Comparison of virtual patient simulation with mannequin-based simulation for improving clinical performances in assessing and managing clinical deterioration: randomized controlled trial. J Med Internet Res 2014;16(9):e214 [FREE Full text] [CrossRef] [Medline]
    16. Durmaz A, Dicle A, Cakan E, Cakir Ş. Effect of screen-based computer simulation on knowledge and skill in nursing students' learning of preoperative and postoperative care management: a randomized controlled study. Comput Inform Nurs 2012 Apr;30(4):196-203. [CrossRef] [Medline]
    17. Cook DA, Erwin PJ, Triola MM. Computerized virtual patients in health professions education: a systematic review and meta-analysis. Acad Med 2010 Oct;85(10):1589-1602. [CrossRef] [Medline]
    18. Liaw SY, Chan SW, Scherpbier A, Rethans JJ, Pua GG. Recognizing, responding to and reporting patient deterioration: transferring simulation learning to patient care settings. Resuscitation 2012 Mar;83(3):395-398. [CrossRef] [Medline]
    19. Cook DA, Garside S, Levinson AJ, Dupras DM, Montori VM. What do we mean by web-based learning? A systematic review of the variability of interventions. Med Educ 2010 Aug;44(8):765-774. [CrossRef] [Medline]
    20. Smith PL, Ragan TJ. Instructional design. Hoboken NJ: Wiley; 2005.
    21. Liaw SY, Scherpbier A, Klainin-Yobas P, Rethans JJ. Rescuing A Patient In Deteriorating Situations (RAPIDS): an evaluation tool for assessing simulation performance on clinical deterioration. Resuscitation 2011 Nov;82(11):1434-1439. [CrossRef] [Medline]
    22. Wang YS, Wang HY, Shee DY. Measuring e-learning systems success in an organizational context: Scale development and validation. Computers in Human Behavior 2007 Jul;23(4):1792-1808. [CrossRef]
    23. Woo Y, Agostinho S, Reeves TC. Implementing authentic tasks in web-based learning environments. EDUCAUSE Quarterly 2007 Jul 30:3.
    24. Herrington J, Reeves TC, Oliver R. A guide to authentic e-Learning. New York: Routledge; 2010.
    25. Keller JM, Suzuki K. Learner motivation and e-learning design: A multinationally validated process. Journal of Educational Media 2004;29(3):229-239.
    26. Marshall S. The use of cognitive aids during emergencies in anesthesia: a review of the literature. Anesth Analg 2013 Nov;117(5):1162-1171. [CrossRef] [Medline]
    27. Kolb DA. Experiential Learning: Experience as the sources of learning and development. New Jersey: Prentice-Hall; 1984.
    28. Georg C, Zary N. Web-based virtual patients in nursing education: development and validation of theory-anchored design and activity models. J Med Internet Res 2014;16(4):e105 [FREE Full text] [CrossRef] [Medline]
    29. Consorti F, Mancuso R, Nocioni MA, Piccolo A. Efficacy of virtual patients in medical education: A meta-analysis of randomized studies. Computers & Education 2012 Nov 01;59(3):1001-1008. [CrossRef]
    30. Salminen H, Zary N, Björklund K, Toth-Pal E, Leanderson C. Virtual patients in primary care: developing a reusable model that fosters reflective practice and clinical reasoning. J Med Internet Res 2014;16(1):e3 [FREE Full text] [CrossRef] [Medline]
    31. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA 2008 Sep 10;300(10):1181-1196. [CrossRef] [Medline]
    32. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Instructional design variations in internet-based learning for health professions education: a systematic review and meta-analysis. Acad Med 2010 May;85(5):909-922. [CrossRef] [Medline]
    33. Norman G. Research in clinical reasoning: past history and current trends. Med Educ 2005 Apr;39(4):418-427. [CrossRef] [Medline]
    34. Youngblood P, Harter PM, Srivastava S, Moffett S, Heinrichs WL, Dev P. Design, development, and evaluation of an online virtual emergency department for training trauma teams. Simul Healthc 2008;3(3):146-153. [CrossRef] [Medline]
    35. Hansen MM. Versatile, immersive, creative and dynamic virtual 3-D healthcare learning environments: a review of the literature. J Med Internet Res 2008;10(3):e26 [FREE Full text] [CrossRef] [Medline]
    36. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011;13(4):e126 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    ABDCDE: Airway, Breathing, Circulation, Disability, and Expose/Examine
    ANCOVA: analysis of covariance
    ELSS: e-learning systems success
    ISBAR: Identity, Situation, Background, Assessment, and Recommendation
    RAPIDS: Rescuing a Patient in Deteriorating Situations
    RCT: randomized controlled trial
    RN: registered nurse
    WBL: Web-based learning


    Edited by G Eysenbach; submitted 11.09.14; peer-reviewed by M Mancini, S Erdley; comments to author 25.09.14; revised version received 15.10.14; accepted 06.11.14; published 12.01.15

    ©Sok Ying Liaw, Lai Fun Wong, Sally Wai-Chi Chan, Jasmine Tze Yin Ho, Siti Zubaidah Mordiffi, Sophia Bee Leng Ang, Poh Sun Goh, Emily Neo Kim Ang. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 12.01.2015.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.