Published on 08.04.15 in Vol 17, No 4 (2015): April
3D Immersive Patient Simulators and Their Impact on Learning Success: A Thematic Review
Background: Immersive patient simulators (IPSs) combine the simulation of virtual patients with a three-dimensional (3D) environment and, thus, allow an illusionary immersion into a synthetic world, similar to computer games. Playful learning in a 3D environment is motivating and allows repetitive training and internalization of medical workflows (ie, procedural knowledge) without compromising real patients. The impact of this innovative educational concept on learning success requires review of feasibility and validity.
Objective: It was the aim of this paper to conduct a survey of all immersive patient simulators currently available. In addition, we address the question of whether the use of these simulators has an impact on knowledge gain by summarizing the existing validation studies.
Methods: A systematic literature search via PubMed was performed using predefined inclusion criteria (ie, virtual worlds, focus on education of medical students, validation testing) to identify all available simulators. Validation testing was defined as the primary end point.
Results: There are currently 13 immersive patient simulators available. Of these, 9 are Web-based simulators and represent feasibility studies. None of these simulators are used routinely for student education. The workstation-based simulators are commercially driven and show a higher quality in terms of graphical quality and/or data content. Out of the studies, 1 showed a positive correlation between simulated content and real content (ie, content validity). There was a positive correlation between the outcome of simulator training and alternative training methods (ie, concordance validity), and a positive coherence between measured outcome and future professional attitude and performance (ie, predictive validity).
Conclusions: IPSs can promote learning and consolidation of procedural knowledge. The use of immersive patient simulators is still marginal, and technical and educational approaches are heterogeneous. Academic-driven IPSs could possibly enhance the content quality, improve the validity level, and make this educational concept accessible to all medical students.
J Med Internet Res 2015;17(4):e91
One key factor of clinical education is the transfer of declarative knowledge (ie, “what to do”) into procedural knowledge (ie, “how to do”). It is performed most effectively in small groups accompanied by a medical teacher . However, in the daily practice clinical instruction is often based on a traditional apprenticeship process and on the expectation that if a student spends enough time in a clinical environment he or she will eventually “get it”. Although this education is good clinical practice and known to be successful, it is impaired by increasing workload of hospital doctors, restrictive working time directives, and changes of students’ attitudes and expectations in the sense of the Generation Y [ ]. Therefore, new educational strategies (eg, skills labs or mannequin simulators) are developed [ , ]. Virtual patient simulators go one step further as they allow case-based learning on personal computers. In a blended learning context they can have a positive impact on knowledge gain [ ]. The available simulators vary greatly in realism and interaction grade, however it is questionable whether this factor affects learners’ outcome. Technological advances have allowed virtual patient simulators, such as immersive patient simulators (IPSs), with a high level of realism and interaction grade similar to computer games (ie, serious games). IPSs must not necessarily be installed on the user’s home computer (ie, workstation based). New technologies enable streaming of even complex programs via the Internet directly into the user’s browser (ie, Web-based). Web-based IPSs are characterized by representation of a three-dimensional (3D) virtual environment via the browser where users can freely interact in real time with their surroundings, and thus become part of the synthetic world, individually or in virtual teams (see ).
The user can playfully immerse himself/herself into the digital environment and faces the consequences of different decisions (ie, trial and error) without putting real patients at risk. By repetitive and playful training of medical procedures, procedural knowledge can be internalized and consolidated. Therefore, IPSs potentially allow time- and location-independent learning and an effective preparation for bedside teaching. However, evaluation of the impact of IPSs on knowledge gain is a demanding task, as gain in procedural experience can hardly be objectivized and is influenced by many educational factors. For medical teachers, information about validity and usability are essential parameters for a possible implementation of such simulators in the current medical curriculum. Furthermore, development of such a simulation is time- consuming and cost-intensive and only worthwhile when there is a proven effect on knowledge gain.
It was the aim of this paper to give a thematic review of the available immersive patient simulators in virtual worlds (VWs) and to evaluate whether the use of these simulators have an impact on knowledge gain by summarizing the existing validation studies.
A systematic search of literature via PubMed of articles from 1986 to 2014 on IPSs was performed with a focus on education of medical students. IPSs were defined as digital environments that simulate medical workflows and show the characteristics of real-time simulation with free interaction in a 3D setting. As browser technology is developing fast, not only workstation-based simulators, but also Web-based simulators were included. Simulation of one-step procedures (ie, intubation, puncturing) were excluded. Search terms included a combination of “simulation,” “virtual reality,” or “virtual worlds,” and “education” or “training.” Peer-reviewed publications from 1986 until 2014 were included. Eligibility assessment was performed independently by two reviewers and disagreements between reviewers were resolved by consensus with a third reviewer. Titles and abstracts of all articles were screened with regard to relevance and consequently grouped—relevant articles and articles of unknown relevance were screened in full text. Furthermore, reference lists of relevant articles were searched for additional articles that were possibly not identified until then. Identified immersive patient simulators were classified by the field of application, technical parameters (ie, Web or workstation based, immersion grade, features), medical content (low ≤3 cases, medium = 4 to 8 cases, high≥9 cases), and existing validation studies.
Validation was assessed according to the consensus guidelines for validation of virtual reality surgical simulators : (1) concordance (aka, face) validity refers to the degree of resemblance between simulator training and training in reality, (2) content validity refers to the degree to which simulated content covers the dimension of the construct it aims to educate, (3) construct (aka, contrast) validity describes the impact of existing knowledge on simulator performance, and (4) predictive validity describes the simulator impact on future performance applications [ , ]. Validation studies were further classified in accordance with the Cochrane Handbook for Systematic Reviews of Interventions. Studies were parameterized by methodical aspects (ie, study design, level of content), technical details of the IPS, end points, and study results. Validity of observational studies was assessed by using the methodological index for nonrandomized studies [ ]. Literature review and data extraction was performed by two reviewers independently, and compared afterwards.
The systematic literature search () identified 16,946 publications that matched the criteria.
A total of 13 publications were identified as relevant in terms of describing the use of virtual worlds in medical education (). Of these publications, 9 described IPSs that were Web-based and 4 described IPSs that were workstation based.
|Virtual world||Type||Content||Immersion||Content level||Reference|
|Second Health London||WB||Emergency medicine||Medium||Medium|||
|CliniSpace||WB||Emergency medicine||Very high||High|||
|3D Emergency Department||WB||Emergency medicine||Medium||High|||
|Inmedea||WB||Various clinical cases||Low||High|||
|Project TOUCH||CBb||Various clinical cases||High||Medium|||
|Virtual Emergency Department||CB||Emergency medicine||High||High|||
|Burn Center||CB||Burn wounds||High||Medium|||
bComputer-based (CB) (aka, workstation based).
Out of 9 Web-based IPSs, 5 of them use the 3D technology of the widely used social network, Second Life (SL). All of these 5 IPSs were developed by university groups. None of these IPSs are routinely used in the education of medical students or medical staff, as they are technical feasibility projects with general medical content and focus on medical knowledge exchange . However, they show the technical feasibility of providing medical education via virtual worlds [ ]. A total of 3 of the Second Life-embedded virtual worlds were designed for team-based training in medical education—Second Health London is provided by the Imperial College London and was used in a pilot feasibility study including a first validation [ ]. MeRiTS also offers team-based training in an SL environment, which was summarized in a feasibility study [ ] without validation so far. Due to the technical specifications of Second Life, the graphical quality is not state of the art. This limitation potentially influences learners’ immersion, as graphical quality and presentation have an effect on immersion grade [ ].
Web-based simulators are comparable to SL simulators, however, the main difference is the technical background. Web-based simulators use a proprietary 3D engine program, which makes development of 3D worlds more resource intensive, but enables more possibilities in graphical quality and custom program design. Of the 4 remaining Web-based IPSs, 1 was already presented in 2003 and uses a custom-made 3D world. Again, the key aspect of the simulator is a feasibility study, in this case particularly with regard to distance learning [, ]. The 3 remaining Web-based IPSs were built by commercial software companies. Of these, 2 of them show a high 3D graphical quality and, thus, a high immersion grade. They offer team- and case-based training of acute clinical cases, with the main focus on emergency medicine [ ]. The high quality is reflected by the elevated pricing of these simulations—more than US $5000 for a 1-year subscription. The medical content is high and custom-made virtual cases can be added for an additional fee. The CliniSpace IPS has already been used by Stanford University for emergency procedure training in the trauma room [ , ]. The remaining IPS, Inmedea, offers low immersion as it does not use a 3D engine, but consists of freeze-frames with drawn graphics. However, the amount of medical content in this simulator is very high, and custom cases can also be added when necessary. Although it is commercially driven, there are several universities that already use this simulator in a blended learning concept [ , ], and there are already first validation studies [ ].
Workstation-based IPSs need to be installed on a computer and offer high immersion with high-fidelity 3D graphics, as they are not limited by the technical handicaps of Internet broadcasting. Out of the 4 simulators, 3 of them are still undergoing feasibility studies, as they were not already validated for their effectiveness [, ]. Although 1 of the simulators was introduced 10 years ago, it is neither routinely used in student education, nor has it been validated yet [ ]. The remaining commercial VW environments by the University of Birmingham offer training in triage casualties in a simulated live exercise [ ], which was part of the first validation study.
The literature search uncovered 5 articles that included validation studies, however, no study assessed all levels of validity. One of the available VW simulators validated the coherence between simulated content and real content (ie, content validity) in training of emergency procedures . The correlation between the outcome of simulator training and alternative training methods was proved in 2 virtual worlds [ , ] (ie, concordance validity)—in all 3 validation studies, simulator training was comparable to alternative training in terms of outcome. The coherence between measured outcome and future performance (ie, predictive validity) was validated in 2 simulators [ , ] and showed a positive correlation. The simulators and corresponding validation studies are summarized in .
|Validity type and study||Number of participants||Virtual world||Reference|
|Cohen et al, 2013||23||Second Health London|||
|LeRoy Heinrichs et al, 2008||30||3D Emergency Department|||
|Youngblood et al, 2008||30||Virtual Emergency Department|||
|Funke et al, 2012||116||Inmedea|||
|Knight et al, 2010||91||Triage Trainer|||
This paper gives an overview of the available IPSs and corresponding validation studies and, therefore, summarizes the current situation in learning in virtual worlds. Previous studies revealed that using virtual patient simulators can have a positive impact on learning success , whereas the effect of realism grade on knowledge gain is still under discussion. IPSs as a subgroup of virtual patient simulators offer high interaction grade and realism as in computer games (ie, serious games). However, we intentionally separated the group of immersive patient simulators from serious games, as the focus of these simulators lies more on knowledge transfer than on classical game elements. The potential of IPSs lies in the internalization of diagnostic and therapeutic procedures, such as resuscitation or diagnosis patterns. Ideally, the underlying procedures are already predefined to achieve performance uniformity, similarly to standard operating procedures in the clinical daily routine [ ]. It is known that immersion plays a fundamental role in virtual reality simulators, as identification with the avatar influences motivation and improves learning success [ ]. However, immersion grade is influenced by many factors [ ] and, therefore, hard to verify. A study from 1999 revealed that “there were no statistical differences in presence or reality judgment between a high-impact workstation and a PC workstation” [ ]. However, this study was conducted before the high-fidelity graphic era and the results are not entirely transferable, as the high-impact workstation (Silicon Graphics) from 1999 was still far away from the 3D capabilities of current standard personal computers, and realism grade was comparably poor. Furthermore, the majority of the students were not used to computer worlds and, thus, less susceptible to learning with 3D worlds. Newer studies revealed that there was a positive impact of high-fidelity visual presentation on degree of immersion [ ] and even on learning performance [ ]. For clinical teachers, the question arises whether IPSs can support the daily routine and have an effect on students’ future performance. Assessment of new educational concepts includes different forms of validity. Content validity describes the correlation between simulated and real content. There was a positive correlation between IPS learning and the standard clinical curriculum when training students on a well-defined procedure like triage training [ ] or basic procedural workflow in emergency patterns [ , ]. In times of limited time resources in the daily clinical workflow, it is desirable that the clinical curriculum be effective both for teachers and students. IPSs can facilitate students’ preparation as they allow time- and location-independent learning at an individual learning pace and with repetitions, which ensure the attainment of a similar knowledge level by the participating students. Verification of predictive validity illustrates that there is an impact of learning with IPSs on future performance, and consequently enables successful preparation for hands-on training [ , ]. However, all current IPSs are used in the blended learning context, as postprocedural review of students’ performance immediately after training is known to be essential for an adequate knowledge gain [ ]. Moreover, there is evidence that preexisting knowledge has a positive impact on simulator performance (ie, construct validity), although the group size of this study was low [ ]. None of the articles assessed all forms of validity. As the studies are heterogeneous in medical content, the number of participants, and type of simulators, the initial question of whether the use of IPSs in the daily curriculum is beneficial in terms of learning success cannot be fully answered at this time. However, validation of single parameters revealed that IPSs can potentially support clinical teaching, although teachers must be aware of the limitations. IPSs are limited in terms of teaching declarative and procedural knowledge. Clinical education is not limited to teaching standard operating procedures, but is furthermore characterized by weighing clinical findings, evaluation of different hypotheses, and clinical experience. Therefore, IPSs are not intended to replace clinical teachers, but should support young students without relevant clinical experience. Regarding content, currently available virtual reality patient simulators range from small procedures, such as triage training, to complex procedures, such as emergency room protocols, up to teamwork with user-user interaction. The more complex the simulation, the more resources are needed. Therefore, it is not surprising that the complex simulations are commercially driven. Academic teachers can rent these simulators, making them more cost-effective, but it is questionable whether the development of a teaching method should be delegated. A highly immersive, multiuser, virtual reality simulator developed and supervised by an expert team of academic teachers would potentially allow, not only that the multiple users would train on one case (ie, team play), but also that one student would train on more than one patient at a time (ie, multitasking). These nontechnical skills can hardly be taught in reality, but are recognized as potential risk factors in high-risk environments like the emergency room or operating room [ ]. It is the responsibility of universities and teaching hospitals to enable teaching methods that improve patient safety, reduce errors, and to further validate these new teaching methods.
Immersive patient simulators can potentially promote learning and consolidation of procedural knowledge. Web-based simulators allow time- and location-independent learning at an individual pace. The use of immersive patient simulators is still marginal, and technical and educational approaches are heterogeneous. Academic-driven IPSs could possibly enhance the content quality, improve the validity level, and make this educational concept accessible to all medical students. The development and validation of such a simulator will be the subject of our future research.
Conflicts of Interest
- Sutkin G, Wagner E, Harris I, Schiffer R. What makes a good clinical teacher in medicine? A review of the literature. Acad Med 2008 May;83(5):452-466. [CrossRef] [Medline]
- Schlitzkus LL, Schenarts KD, Schenarts PJ. Is your residency program ready for Generation Y? J Surg Educ 2010;67(2):108-111. [CrossRef] [Medline]
- Zendejas B, Brydges R, Wang AT, Cook DA. Patient outcomes in simulation-based medical education: a systematic review. J Gen Intern Med 2013 Aug;28(8):1078-1089 [FREE Full text] [CrossRef] [Medline]
- Gerdes B, Hassan I, Maschuw K, Schlosser K, Bartholomäus J, Neubert T, et al. [Instituting a surgical skills lab at a training hospital]. [Article in German]. Chirurg 2006 Nov;77(11):1033-1039. [CrossRef] [Medline]
- Cook DA, Erwin PJ, Triola MM. Computerized virtual patients in health professions education: a systematic review and meta-analysis. Acad Med 2010 Oct;85(10):1589-1602. [CrossRef] [Medline]
- Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK, Hanna GB, Work Group for Evaluation and Implementation of Simulators and Skills Training Programmes. Consensus guidelines for validation of virtual reality surgical simulators. Surg Endosc 2005 Dec;19(12):1523-1532. [CrossRef] [Medline]
- Graafland M, Schraagen JM, Schijven MP. Systematic review of serious games for medical education and surgical skills training. Br J Surg 2012 Oct;99(10):1322-1330. [CrossRef] [Medline]
- Gallagher AG, Ritter EM, Satava RM. Fundamental principles of validation, and reliability: rigorous science for the assessment of surgical education and training. Surg Endosc 2003 Oct;17(10):1525-1529. [CrossRef] [Medline]
- Slim K, Nini E, Forestier D, Kwiatkowski F, Panis Y, Chipponi J. Methodological index for non-randomized studies (minors): development and validation of a new instrument. ANZ J Surg 2003 Sep;73(9):712-716. [Medline]
- Boulos MN, Hetherington L, Wheeler S. Second Life: an overview of the potential of 3-D virtual worlds in medical and health education. Health Info Libr J 2007 Dec;24(4):233-245. [CrossRef] [Medline]
- Kamel Boulos MN, Ramloll R, Jones R, Toth-Cohen S. Web 3D for public, environmental and occupational health: early examples from second life. Int J Environ Res Public Health 2008 Dec;5(4):290-317 [FREE Full text] [Medline]
- Chodos D, Stroulia E, King S. MeRiTS: simulation-based training for healthcare professionals. Stud Health Technol Inform 2011;163:125-131. [Medline]
- Cohen D, Sevdalis N, Patel V, Taylor M, Lee H, Vokes M, et al. Tactical and operational response to major incidents: feasibility and reliability of skills assessment using novel virtual environments. Resuscitation 2013 Jul;84(7):992-998. [CrossRef] [Medline]
- Dev P, Heinrichs WL, Youngblood P. CliniSpace: a multiperson 3D online immersive training environment accessible through a browser. Stud Health Technol Inform 2011;163:173-179. [Medline]
- Creutzfeldt J, Hedman L, Heinrichs L, Youngblood P, Felländer-Tsai L. Cardiopulmonary resuscitation training in high school using avatars in virtual worlds: an international feasibility study. J Med Internet Res 2013;15(1):e9 [FREE Full text] [CrossRef] [Medline]
- LeRoy Heinrichs W, Youngblood P, Harter PM, Dev P. Simulation for team training and assessment: case studies of online training with virtual worlds. World J Surg 2008 Feb;32(2):161-170. [CrossRef] [Medline]
- Funke K, Bonrath E, Mardin WA, Becker JC, Haier J, Senninger N, et al. Blended learning in surgery using the Inmedea Simulator. Langenbecks Arch Surg 2013 Feb;398(2):335-340. [CrossRef] [Medline]
- Creutzfeldt J, Hedman L, Medin C, Heinrichs WL, Felländer-Tsai L. Exploring virtual worlds for scenario-based repeated team training of cardiopulmonary resuscitation in medical students. J Med Internet Res 2010;12(3):e38 [FREE Full text] [CrossRef] [Medline]
- Caudell TP, Summers KL, Holten J, Hakamata T, Mowafi M, Jacobs J, et al. Virtual patient simulator for distributed collaborative medical education. Anat Rec B New Anat 2003 Jan;270(1):23-29 [FREE Full text] [CrossRef] [Medline]
- Youngblood P, Harter PM, Srivastava S, Moffett S, Heinrichs WL, Dev P. Design, development, and evaluation of an online virtual emergency department for training trauma teams. Simul Healthc 2008;3(3):146-153. [CrossRef] [Medline]
- Knight JF, Carley S, Tregunna B, Jarvis S, Smithies R, de Freitas S, et al. Serious gaming technology in major incident triage training: a pragmatic controlled trial. Resuscitation 2010 Sep;81(9):1175-1179. [CrossRef] [Medline]
- Kurenov SN, Cance WW, Noel B, Mozingo DW. Game-based mass casualty burn training. Stud Health Technol Inform 2009;142:142-144. [Medline]
- Gutiérrez F, Pierce J, Vergara VM, Coulter R, Saland L, Caudell TP, et al. The effect of degree of immersion upon learning performance in virtual reality simulations for medical education. Stud Health Technol Inform 2007;125:155-160. [Medline]
- Alverson DC, Saiki SM, Kalishman S, Lindberg M, Mennin S, Mines J, et al. Medical students learn over distance using virtual reality simulation. Simul Healthc 2008;3(1):10-15. [CrossRef] [Medline]
- Wünschel M, Leichtle U, Wülker N, Kluba T. Using a web-based orthopaedic clinic in the curricular teaching of a German university hospital: analysis of learning effect, student usage and reception. Int J Med Inform 2010 Oct;79(10):716-721. [CrossRef] [Medline]
- Horstmann M, Renninger M, Hennenlotter J, Horstmann CC, Stenzl A. Blended E-learning in a Web-based virtual hospital: a useful tool for undergraduate education in urology. Educ Health (Abingdon) 2009 Aug;22(2):269. [Medline]
- Lee CH, Liu A, Del Castillo S, Bowyer M, Alverson D, Muniz G, et al. Towards an immersive virtual environment for medical team training. Stud Health Technol Inform 2007;125:274-279. [Medline]
- Papakonstantinou D, Poulymenopoulou M, Malamateniou F, Vassilacopoulos G. Enabling the use of enhanced medical SOPs by an mLearning training solution. Stud Health Technol Inform 2013;190:86-88. [Medline]
- Kuhlen TW, Hentschel B. Quo vadis CAVE: does immersive visualization still matter? IEEE Comput Graph Appl 2014;34(5):14-21. [CrossRef] [Medline]
- Botella C, Rey A, Perpiñá C, Baños R, Alcañiz M, Garcia-Palacios A, et al. Differences on presence and reality judgment using a high impact workstation and a PC workstation. Cyberpsychol Behav 1999;2(1):49-52. [CrossRef] [Medline]
- Huerta R. Dissertations & Theses, The University of Texas - Pan American. Ann Arbor, MI: UMI Dissertations Publishing; 2012 Aug 01. Measuring the impact of narrative on player's presence and immersion in a first person game environment [MS disseration] URL: http://search.proquest.com/docview/1152549474 [accessed 2015-03-25] [WebCite Cache]
- Welke TM, LeBlanc VR, Savoldelli GL, Joo HS, Chandra DB, Crabtree NA, et al. Personalized oral debriefing versus standardized multimedia instruction after patient crisis simulation. Anesth Analg 2009 Jul;109(1):183-189. [CrossRef] [Medline]
- Catchpole K, Mishra A, Handa A, McCulloch P. Teamwork and error in the operating room: analysis of skills and roles. Ann Surg 2008 Apr;247(4):699-706. [CrossRef] [Medline]
|IPS: immersive patient simulator|
|SL: Second Life|
|VW: virtual world|
Edited by G Eysenbach; submitted 27.04.14; peer-reviewed by M Troconis, S Edelbring, SY Liaw; comments to author 13.12.14; revised version received 17.01.15; accepted 03.02.15; published 08.04.15
©Robert Kleinert, Roger Wahba, De-Hua Chang, Patrick Plum, Arnulf H Hölscher, Dirk L Stippel. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 08.04.2015.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.