The definition of realism was how the health care participants felt that the training environment provided by VRS was like their actual clinical environment and medical practice. There were 4 (33%) of 12 studies [27,28,30,33] where the analysis and evaluation of VRS training programs with regard to realism was explicitly conducted. These studies [27,28,30,33] demonstrated the capabilities of VR technology in realistically simulating clinical scenarios and medical practices related to disasters. The confirmatory factorial analysis by Loverglio et al [33] suggested that such the replicability is the main contributing factor in the sense of realism. Here, the health care participants could interact with sequences of actions and tasks required to deal with the in-hospital disaster scenarios [27,28,30]. These findings may indicate that VR technology enables immersive training and learning in the clinical domain. This is consistently found in prior integrated reviews in medicine [23] as well as those in other nonmedical industry domains [12,13].

In a prior integrated review by Miller et al [23], triage and treatment training from human-made MCIs was only for application practices. This limited applicability was mitigated in the current integrated review by including new programs of building evacuation safety training from natural catastrophic events, such as earthquakes [33]. The technological advances in VRS during the past 15 years may be 1 of the contributing factors in the application to clinical scenarios that require sophisticated modeling and environment depiction.

There were complains about the VR interface from an early study by Heinrich et al [27] in 2008, but no such complaints were discovered in recent studies until 2020. This might imply that the technological maturity of VRS is at a level that includes consideration of extraneous variables, such as technological sophistication of the participants and ease of navigation.

In total, 10 (83%) of 12 studies [6,24,26-32,34] (2 studies by Ferra et al [35] and Lovreglio et al [33]) evaluated VRS for its learning effects on in-hospital disaster preparedness training. Learning effects on increased confidence and enhanced knowledge acquisition were captured in all the 10 studies [6,24,26-32,34], and such learning effects were consistently found in previous integrated reviews in medicine [23] as well as other nonmedical industry domains [12].

In a prior integrated review by Miller et al [23], the learning effects were limited in triage and treatment knowledge and skills because all included studies focused on those practices. The larger variation of applied practices was found in our integrated review; it included not only triage practices but also training programs on fire prevention and safety, and emergency evacuation planning and execution. Through this new integrated review, it was found that the learning effects from the use of VRS can be generalized to a variation of medical training practices.

There were 4 (33%) quasi-experimental studies [6,31,32,34] where EG participants with VRS training were analyzed and directly compared with their CG counterparts without VRS training. These studies [6,31,32,34] consistently showed no significant difference between the 2 groups in posttest evaluation on self-efficacy or knowledge acquisition. The EG participants, however, showed statistical or clinical performance enhancement in gained knowledge and task completion efficiency when evaluated in final live drills (eg, performing the correct sequence of steps in extinguishing a simulated fire) [32]. This result suggested the practical usefulness of VRS and its learning effects on real disaster scenarios that health care professionals may encounter in the future.

One of the important learning effects is increased learning retention [36,37]. The prior integrated review by Miller et al [23], however, could not derive any findings on it due to a lack of studies. In this integrated review, there was 1 (8%) study [31] that took the learning retention effect into consideration. The study [31] conducted a longitudinal experiment with repeated measures taken during a year, and the results showed improved knowledge gaining as training progressed. This statement, however, is somewhat difficult to be generalized due to the insufficient volume of longitudinal experimental studies. This insufficiency may be, in part, attributed to the fact that inputs and efforts, including participants, facilities, and program management, are much greater and have to be maintained for a longer period when compared to 1-shot studies.

Cost analysis of the training program implementation and operation is 1 of the important factors training bodies have to consider, but it was not part of the previous integrated review by Miller et al [23]. In this integrated review, 2 (16%) of 12 relevant studies [26,35] suggested consistent findings for the cost advantage in implementing and operating VRS training programs over the live drill alternative. The larger initial investment from VRS training was undeniable, but it could be scalable and compensated for its longer-term use with the large number of participants. A comparative numerical study by Farra et al [35] found that the initial additional cost from VRS training was $97.99 when compared to the live drill alternative, and this initial investment provided a benefit of $114.65 after 3 years of program operation. Farra et al [35] attributed this to the repeatability feature of VRS anywhere and anytime, enabling just-in-time training.

This integrated review of 12 studies [6,24,26-35] showed the applicability of VRS to different training scenarios of in-hospital disaster preparedness, including triage and transport of victim patients, emergency treatment, fire extinguishing, and building evacuation. These, however, do not entirely cover all disaster scenarios that could happen in hospitals. Investigating VRS with other challenging clinical scenarios that require sophisticated environment depiction and complicated training tasks (eg, patient protection and evacuation in scenarios of water entering a hospital due to floods) is an interesting future direction for effective in-hospital disaster management. The recent maturity of VR technologies in modeling and rendering and the development tools would be able to address the complexity of different clinical scenarios.

The number of VRS studies on in-hospital disaster preparedness training programs is still limited. Compared to the previous integrated review by Miller et al [23], the number increased from 10 to 12. Considering that Miller et al [23] collected all the VRS studies, including training scenarios occurring outside hospitals and in-hospital events, the increase in VRS studies on in-hospital training programs is noticeable. In addition, it was observed that the number of studies published during the past 5 years (from 2016 to 2020) was greater than that published in the older but longer period between 2006 and 2015 (ie, 7 [29-35] vs 5 studies [6,24,26-28]).

The scale of participants is 1 of the important variables to draw conclusive findings from studies. There were 4 (33%) of 12 studies [6,31,33,35] that considered the scale of participants by conducting VRS training programs with more than 50 participants. This increased from 2 in the previous integrated review by Miller et al [23] and was double. In addition, all 4 studies [6,31,33,35] were conducted in the recent 5 years. These observations suggest that future studies would put more emphasis on this important variable and the potential outcomes can be more conclusive.

It was observed that 7 (58%) of 12 studies [29-35] since the past 5 years used consistent development platforms, such as the Virtual Electrosurgical Skill Trainer (VEST) and Unity, for the clinical implementation of VRS training programs. The use of established development platforms enables rapid implementation, easy scenario replication, and, therefore, increased feasibility in conducting VRS-based clinical training studies. This observation can somewhat explain the increasing trends in the number of VRS studies and studies with large-scale participants.

It was also observed that the details of evaluation processes were not fully described. There were only 3 (25%) of 12 studies [24,28,30] that explicitly stated the skill levels and experiences of health care participants, and 1 (8%) study [26] even lacked details of the participant number. Demographic details can be 1 of the influential variables to allow in-depth interpretation of outcomes from studies (eg, how learning effects work for different levels of expertise, as shown in the study by Pucher et al [28]). Similarly, qualitative analysis in the participants’ experience was described using Liker-type surveys, postexperience interviews, and focus group open discussions, where there were little details on the reliability and validity of coding of qualitative data from interviews or focus groups. Such a lack was consistently found in the previous integrated review by Miller et al [23]. This integrated review, therefore, suggests that process details need to be sufficiently described in future studies, which will facilitate future investigators to draw deep insights into the value of VRS.

It was noted that the baseline performance comparisons of different studies highly varied: 3 (25%) of 12 studies compared VRS training programs among participants where some had previous disaster preparedness training experiences prior to study enrollment; 6 (50%) studies [6,26,27,30-32,34,35] compared VRS training effects against the experiences of live drills; and the remaining 3 (25%) studies [28,29,33] did not explicitly state the comparison approach. This may suggest that some efforts on developing the standard of VRS training program comparison protocols are meaningful, and this will allow for objective and quantitative comparison among different studies and draw general insights that are applicable to different disaster training programs.