An electronic literature search was carried out in the Cochrane Library, Web of Science, PubMed, Embase, and CINAHL (Cumulative Index to Nursing and Allied Health Literature) from their inception to December 2019. The search strategies used in PubMed, Embase, and the Cochrane Library are listed in the Multimedia Appendix 1. Slightly modified search strategies were used in the other databases. Additionally, we manually examined reference lists of the selected articles to retrieve other relevant publications. Two investigators conducted searches independently, and EndNote software was used to import and manage selected documents.

This study included randomized controlled trials (RCTs) or trials employing quasi-experimental randomized design, including those in the form of dissertations and conference papers, based on the PICO (Population–Intervention –Comparison–Outcome) method. In this study, the PICO elements were as follows:

Two reviewers (FQC and YFL) independently extracted information based on preset standards, including authors, publication date, nation, sample size, participants type, research project, intervention regimens, and outcomes.

Two reviewers (FQC and YFL) assessed the studies’ quality independently by referring to the Cochrane Handbook for Systematic Reviews of Interventions [36], which includes 7 domains corresponding to a specific type of bias: (1) random sequence generation (selection bias); (2) allocation concealment (selection bias); (3) blinding of participants and personnel (performance bias); (4) blinding of outcome assessment (detection bias); (5) incomplete outcome data (attrition bias); (6) selective reporting (reporting bias); and (7) other biases. A judgement of “low risk,” “high risk,” or “unclear risk” of bias was assigned to each domain. When disagreements between reviewers could not be resolved through discussion, two additional reviewers (ZLS and JFG) made the final decision.

The meta-analysis was conducted using RevMan 5.3 [37], a desktop version of Review Manager software used for Cochrane intervention and flexible reviews. For continuous data, we reported standard mean difference (SMD) with 95% confidence intervals. In each analysis, I² was used to measure the statistical heterogeneity among studies. According to the values of P and I², the fixed-effect model (P>.1, I²<50%) or random-effects model (0<P<.1, I²≥50%) were selected [38].

A total of 2716 potential studies were identified from 5 databases (n=2712) and relevant references (n=4). After removing 1072 duplicates, the remaining articles were reviewed and those that did not meet the inclusion criteria were excluded. A total of 1644 articles were screened by title and abstract, of which 1581 articles were excluded. A total of 63 full-text articles were downloaded and assessed, from which 51 were excluded. Finally, 12 studies, including 821 participants, were selected for this study. A flow chart of the study selection process is presented in Figure 1.

Studies included trials conducted in 7 countries: United States [31,39-42], Turkey [43], Canada [44], Korea [45], Singapore [46], Portugal [47], and China [48,49]. Two trials adopted a 3-arm group design [42,45], while 10 trials used a 2-arm group design. Sample sizes ranged from 20 to 172 participants. In all trials, participants were nursing students, except for one study in which participants were nursing staff [49]. Six of 12 trials compared VR education with traditional education [31,39,41,42,48,49], while the remaining trials compared VR education with other simulation types including fidelity manikin [44,47], mannequin-based simulation [40,45,46], and plastic model [43]. The characteristics of the participants, intervention details, and outcome measures are presented in Table 1. Supplementary information of intervention in experimental and control conditions is shown in Multimedia Appendix 2.

Based on the Cochrane criteria, a risk-of-bias assessment is presented in Figures 2 and 3. Four of 12 studies reported randomized methods in detail [41,43,46,47], while the remaining 8 trials did not provide the methods of sequence generation. None of the trials provided concealment methods, except for one that reported the use of anonymization [47]. In all trials, no blind method was used on participants due to the particularity of the intervention methods. Two trials reported employing blinding of assessors [39,43]. Additionally, 2 studies mentioned dropouts without detail on handling information [46,49].

For the outcome of knowledge, VR education showed more effectiveness on nursing education than traditional education or other simulation education methods. A qualitative study on VR use in nursing education also concluded that, through the concrete experience of the virtual patient simulation and the reflection tool, students could understand what they were taught and how to utilize the new knowledge [50]. Additionally, a previous study, which focused on virtual reality for health professions education, indicated that VR with higher interactivity showed more effectiveness for knowledge [21]. These studies support the fact that an interactive learning environment encourages students to establish connections between concepts [51]. Most of the studies included in our meta-analysis used interactive VR education methods, which could explain the results.

Our results found no significant difference between VR education and other education methods for the outcome of skills, which seems to be in line with a previous systematic review [34]. The review concluded that virtual reality groups performed comparably to simulation groups in skill performance scores and skill success rate [34]. In our study, all the included trials that reported skills employed other simulation education methods in control groups. Similarly, we concluded that VR was not more effective in improving skills than other simulation methods in nursing education. A possible reason for these results is that there is a gap between completing virtual cases and real practice. Nursing skills learned on a virtual platform may not be transferable to real situations effectively because of the immaturity of VR technology [48].

There was no significant difference on participants’ satisfaction between VR education and education methods in control groups. High heterogeneity was found. Through sensitive analysis, we found that 1 of the 4 included studies showed that VR was more satisfactory [47]. In one trial conducted in 2012, some participants pointed out the immaturity of VR technology affecting users’ satisfaction [45]. In contrast, 2 studies in recent years showed no difference between the 2 groups [31,40]. Thus, we consider that participants’ satisfaction with VR education may vary according to technical conditions. Although in the 21st century nursing students had already shown high levels of usefulness, ease, and intention to use clinical VR simulation, VR is not widely used in nursing education [52]. With the progress of technology, VR can better satisfy the users. However, further research is needed to confirm our results.

The results in confidence indicated no difference between experimental and control conditions. VR could not enhance the confidence of participants more effectively than control conditions, which was consistent with a previous study from Korea [53]. When VR was used for operation exercises, it was often necessary to use a mouse at the same time [53]. Thus, the operation method is more difficult in VR when compared with other simulations such as the manikin.

We also conducted a meta-analysis of performance time. The results suggested that VR was not more effective on reducing performance time than other educational methods. We found large heterogeneity among studies, even when a sensitivity analysis was conducted by using the leave-one-out method. The observed heterogeneity may be due to the different research designs of the selected studies, such as operation projects, VR devices, and education methods in control groups. One study on the effectiveness of VR endoscopy simulation training analyzed performance time with sufficient data and found no difference between VR and control groups; however, the quality of the evidence was very low [54]. In contrast, a study conducted in clinical medicine found that VR can help operators shorten performance time [55]. Therefore, more experiments are needed in the future to study the effectiveness of VR on performance time in nursing education.

Our study has the following strengths. First, our study is the first meta-analysis assessing the impact of VR on nursing education. Second, to assess the effectiveness of VR education, we evaluated 5 outcome measurements—knowledge, skills, satisfaction, confidence, and performance time—which can probably provide reference for nursing education.

There are also some limitations in our study. First, we only included articles published in English, which may affect the results of meta-analysis. Second, some of the included studies failed to provide the details of sequence generation, allocation concealment, and blinding methods. Third, we included 12 studies that have different interventions in control groups, which may cause significant heterogeneity among the studies.

This meta-analysis provides a comprehensive evaluation of the use of VR on nursing education. We found that VR education methods can improve nursing students’ knowledge. However, for the outcomes of skills, satisfaction, confidence, and performance time, there seems to be no difference between VR and other education methods. In general, the use of VR should be considered to enhance knowledge and as a complement of other simulation strategies to improve the quality and safety of clinical practice. However, the heterogeneity and risk of bias among the included studies should be taken into consideration. Rigorously designed large-scale studies are required to further confirm the results in this review.