This study follows the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines [23]. Multimedia Appendix 1 provides a PRISMA checklist of this meta-analysis. The protocol was registered in PROSPERO (CRD42023466622).

This study included randomized controlled trials (RCTs) from inception to August 2024. Inclusion and exclusion criteria are listed in Textbox 1.

The four outcomes of interest were defined as follows: (1) knowledge: remembering and understanding basic concepts, measured by question-based theoretical tests [24]; (2) procedural skills: following a series of steps to accomplish certain tasks, demonstrated by simulation exams [24]; (3) clinical reasoning: evaluating and reacting to clinical situations, demonstrated by simulation examinations [24]; and (4) communication skills: Interacting with patients or colleagues verbally and nonverbally, demonstrated by simulation examinations [24].

A thorough search was carried out in the Cochrane Library, PubMed, Embase, and Educational Resource Information Centre databases by specifying the desired population, intervention, and comparison. Search terms were selected judiciously, with specific terms customized for each database (Multimedia Appendix 2). Additionally, we searched Google and reference lists of the selected studies to retrieve other relevant publications.

Search records were imported into the EndNote library (version X9; Clarivate). After eliminating duplicates, the remaining studies underwent eligibility screening by 2 independent reviewers (NJ and SYL) according to predefined inclusion and exclusion criteria (Textbox 1). The initial screening process involved the assessment of titles and abstracts for relevance. Subsequently, full-text screening was performed, and the rationale for exclusion was documented in the PRISMA flowchart. In cases of discrepancies between the 2 reviewers, a third reviewer (SC) was consulted to reach a consensus. Study authors were contacted for crucial missing information. Finally, all researchers agreed on the final literature to be included in the analysis.

Two reviewers (NJ and SYL) independently extracted data in a structured form in Microsoft Excel, including author, publication date, country, participants, sample size, type of intervention, type of comparison, and outcomes. If a study had multiple posttest measurements, only the first measurement was recorded due to concern about the learning effect. If participants were rated by both SPs and independent evaluators, the ratings of the latter were preferred. If an outcome was assessed by multiple rating scales, the scale with the highest intraclass correlation coefficient was chosen. If an outcome was reported as multiple items instead of a total score, the primary item was selected, and if that was impossible, the mean score of all items was calculated. If a study had both mannequin and RP as control groups, each pairwise comparison was included separately while splitting the shared intervention group approximately evenly among the comparisons. The outcomes were recorded as mean values and SDs. For studies that reported outcomes as median values and IQRs or as mean values and CIs, these measures were converted to mean values and SDs using methods described in previous literature [25,26].

Two reviewers (NJ and SYL) independently assessed the risk of bias of the included studies using the Cochrane tool [25]. The following domains were considered: random sequence generation, allocation sequence concealment, blinding of participants or personnel, blinding of outcome assessment, complete outcome data, selective reporting, and other sources of bias. Discrepancies were resolved by discussion with a third author. Studies were not excluded from data extraction or analysis based on bias assessment scores.

Since data were measured using different tools, the mean differences were recalculated into standardized mean differences (SMDs). The results are displayed in forest plots, with pooled SMDs computed using random-effects meta-analysis models. We interpreted the effect size as small (SMD=0.2), moderate (SMD=0.5), or large (SMD=0.8) [25]. Publication bias was assessed by the Egger regression asymmetry test.

Cochran Q (chi-square test) was used to evaluate statistical heterogeneity, with a statistical significance threshold at P<.10. I² statistic was adopted to quantify the degree of heterogeneity, where I² was categorized as unimportant (0%-40%), moderate (30%-60%), substantial (50%-90%), and considerable (75%-100%) [25]. To investigate possible sources of heterogeneity, the leave-one-out method was first applied. Next, for each outcome, we conducted subgroup analyses by discipline (medicine; nursing), level (undergraduate; nonundergraduate [graduate students or clinical staff]), and comparison (mannequin; RP) to see if I²<50% in both groups [27]. Univariate random-effects meta-regression analyses were then performed to investigate whether heterogeneity between trials could be attributed to year of publication, age of participants, discipline, level, and comparison. Multivariate meta-regression was not performed due to the limited number of studies and the risk of overfitting.

Trial sequential analysis (TSA) was used to evaluate the strength of evidence and adjusted for potential errors. The analysis set specific values for type 1 and 2 errors (5% and 20%) and used these values to calculate trial sequential monitoring boundaries, futility boundaries, and the required information size (RIS) [28]. The mean difference to generate RIS was set to detect a mean difference of 2.0 between intervention and comparison. The variance was estimated by pooling all included trials. Heterogeneity was adjusted based on the estimated ratio between the variance in the random-effects model and the variance in the fixed-effects model. Finally, a graphical evaluation was used to determine if the cumulative Z curve met defined thresholds.

All data analyses were performed using package meta of R software (version 4.2.3; R Foundation for Statistical Computing) [29] and TSA software (version 0.9.5.10 beta; Copenhagen Trial Unit). Two-sided P values<.05 were considered to be statistically significant.

After the selection process, a total of 27 studies with 1480 participants were included (Figure 1). In terms of discipline, participants in 14 studies were from the field of medicine [16,18,30-41]; participants in 11 studies were from the field of nursing [42-52]; and the remaining 2 studies enrolled mixed participants from medicine and nursing [53,54]. In terms of level, participants in 14 studies were undergraduate students [16,31,35,39,42-46,50-54]; participants in 8 studies were graduate students or clinical staff [18,32-34,36-38,40,41,49]; and the remaining 2 studies enrolled mixed undergraduate and nonundergraduate participants [30,48]. In terms of comparison, 13 studies compared VSs and mannequins [33-36,38,40,44-51]; 10 studies compared VSs and RPs [16,18,30,31,37,39,41-43,52-54]; and the remaining study included mannequins and RPs as comparison groups simultaneously [32]. Detailed information and extracted data for each study are summarized in Multimedia Appendix 3 [16,18,30-54].

Following Cochrane guidance [25], the risk of bias was assessed for all the included studies. The results are summarized in Figure 2 [16,18,30-54]. Out of 27 studies, 15 studies described an adequate random sequence generation method; 11 studies did not provide a clear description; and the remaining 1 study allocated participants to intervention and comparison arms based on the order in which they volunteered. Allocation concealment was not explicitly mentioned in most studies except 2. Blinding participants was impossible to avoid in this type of research but should not raise a major concern. Four studies had high-risk performance bias since researchers were not blinded and might give biased ratings; 12 studies were low-risk due to blinding of personnel; the others did not provide a clear description. The risk of detection bias was rated as unclear in most studies except 4, which clearly stated that statistical analysts were blinded. Most studies had low-risk attrition bias because no participants were lost, while participant dropout in 6 studies had an unclear impact on outcome assessment. Due to the absence of protocols, most studies had unclear reporting bias; 7 studies with protocols provided were rated as low risk. Given the difficulty of evaluating other biases (eg, volunteer bias), the risk was judged to be unclear for all studies.

The aim of this meta-analysis was to evaluate the effectiveness of VSs in comparison with mannequins and RPs in medical and nursing education. We found that VSs and conventional simulations were not statistically different in teaching knowledge, procedural skills, clinical reasoning, and communication skills. Among nursing participants, VSs were more effective in improving procedural skills. For the training of communication skills, VSs tended to be increasingly effective with time.

This meta-analysis followed the original protocol of PROSPERO (CRD42023466622). TSA and meta-regression were added to further evaluate the accumulative evidence and identify potential sources of heterogeneity. We believe that these analyses do not negatively affect the reliability of our results; instead, they offer novel insights and strengthen the existing content. Here, we state the aforementioned deviations to ensure our study’s transparency.

There was no significant difference in knowledge gained between VSs and mannequins or RPs. This result aligns with a previous meta-analysis comparing 2D VSs with traditional education methods [21], while other analyses suggested that 3D VSs were more effective [22,55-57]. In our meta-analysis, almost all studies adopted 3D VSs, but the comparison was specifically narrowed down to conventional simulations. Our pooled effect size tended to favor VSs. Indeed, it can be beneficial to use VSs for knowledge delivery. First, digital platforms or software provide repeatable, information-rich feedback for learning [58]. Second, immersive technology (ie, devices that provide a sense of realism and immersion in the computer-generated world through sensory stimuli) such as VR promotes students’ interest with enhanced satisfaction, self-efficacy, and engagement [59]. Although VSs are viable alternatives to mannequins and RPs, undesirable usability may hinder the learning process. For instance, Cobbett and Snelgrove-Clarke [46] reported in their study that nearly half of the students expressed dislike for the VS due to technological issues such as the “online program was slow” and “didn't know where to find things.” Consequently, the development of VSs should consider features that enhance ease of use and users’ level of satisfaction (eg, step-by-step instructions, user-friendly interfaces) so that students allocate their time appropriately on learning instead of wrestling with the platform or software.

There was no significant difference in improving procedural skills between VSs and mannequins or RPs, while our pooled effect size tended to favor mannequins or RPs. Subgroup analyses showed that VSs were less effective among nursing participants. These results were inconsistent with previous meta-analyses comparing VSs with traditional education methods [21,22,57]. Such discrepancies may be due to different definitions of comparison. While VSs are advantageous for practicing and mastering skills relative to didactic teaching, this is not necessarily true if they are compared with mannequins and RPs. A gap exists between digital environments and real circumstances, and the abstraction of procedures may not be transferable to reality [60]. Notwithstanding, VSs are cost-effective alternatives that enable learners to repeat the training freely without worrying about time, space, and patient harm [61]. VSs may be designed to achieve better skill acquisition. First, the addition of haptic technology, for example, offers more realistic hands-on experiences [36]. Second, too much immersion may not be desirable as it can lead to cognitive overload and hamper procedure learning [42].

There was no significant difference in fostering clinical reasoning between VSs and mannequins or RPs. Again, this result does not conform to previous meta-analyses which compared VSs to no intervention or traditional education methods [21,62]. Our pooled effect size tended to favor VSs. It has been suggested that VSs, with a large number of customizable clinical scenarios, are highly suitable for improving clinical assessment and decision-making [63]. In particular, the following elements may be beneficial: a training duration of more than 30 minutes, varied clinical scenarios, nonimmersive 2D digital environments, and postscenario feedback [62]. Similar to procedural skills, nonimmersive VSs appear to be more effective than their immersive counterparts. According to the cognitive load theory. 3D digital environments can increase cognitive load as students get distracted by irrelevant stimuli [62]. More data are still required to reach a conclusive result regarding the comparative effectiveness of VSs in improving clinical reasoning.

There was no significant difference in enhancing communication skills between VSs and mannequins or RPs. This is in line with the findings of a previous meta-analysis comparing digital education to traditional learning [15]. Our pooled effect size tended to favor mannequins or RPs. It is noteworthy, however, that more recent studies tended to favor VSs. A possible explanation is that advances in technology can increasingly overcome the long-standing lack of realism and means for interaction in VSs [30]. Greater immersion can elicit a greater sense of “being there” for users [64]. This psychological experience is critical for communication training, to make learners feel as if they are having face-to-face conversations with digital avatars so that higher levels of empathy can be induced. Moreover, VSs have fewer limits on time or place, eliminate the costs of SP training, and offer a variety of clinical scenarios available for student communication training. These properties render VSs with great potential as novel methods for teaching communication skills.

To the best of our knowledge, this is the first meta-analysis comparing the effectiveness of VSs and conventional simulations. Strengths of this study include pragmatic research questions, well-defined outcome measures, comprehensive searches of the most up-to-date RCTs (ie, the gold standard for evaluating the effectiveness of interventions), and critical analyses of the evidence. Our results indicate that VSs are viable alternatives to face-to-face learning modalities. For institutions, time and money saved can be invested in other research and teaching projects. For students, the removal of time, space, and location limitations leads to greater freedom of learning and wider popularization of education.

On the other hand, limitations must be acknowledged when interpreting the results of this study. First, researchers in several studies were not blinded and might introduce bias when assessing participants. Second, certain subgroup or meta-regression analyses were not feasible or included only one entry due to the limited number of studies. Further analyses may be possible when more RCTs are published. Third, the heterogeneity of the included studies could not be fully explained by leave-one-out, subgroup, or meta-regression analyses. This lack of explanation may be attributed to other unspecified sources of methodological and subject heterogeneity.

Overall, this meta-analysis did not find significant differences between VSs and mannequins or RPs in improving knowledge, procedural skills, clinical reasoning, and communication skills. For procedural skill acquisition, VSs were less effective than mannequins or RPs among nursing participants. For communication skill development, VSs exhibited increasing potency with more advanced technology over time. Given other prominent advantages of VSs (eg, cost-effectiveness, flexibility regarding time or space), it is worth considering these tools as alternatives to traditional education methods. As technology continues to evolve, the comparative effectiveness of VSs will need to be reevaluated and updated. In particular, it is anticipated to witness the integration of emerging large language models (eg, ChatGPT) in VSs and see if they can revolutionize educational practices. We also recommend future research to investigate the effectiveness of VSs across various design configurations.