The purpose of the surgical treatment of pelvic fracture is to maintain the anatomical shape of the pelvis and restore its biomechanical characteristics. When reconstructing acetabular fractures, the basic principles of anatomical reconstruction, stable fixation of the articular surface, and immediate postoperative exercises should be observed. Cimerman et al [25] introduced a computer program for virtual surgery experiments of pelvic and acetabular fractures based on real fracture data; see Cimerman et al [25] for an image of an advanced computerized planning module. Using the 3D viewing tool, the surgeon can build a virtual model of the pelvic fracture. This case study demonstrates the possibility of virtual simulated surgery. The computer program is an easy-to-use application program with great potential for application in clinical practice, teaching, and research. Acetabular fractures are difficult to classify because of the complex 3D anatomical structure of the pelvis; 3D printing is helpful for understanding and reliably classifying acetabular fractures, and 3D VR may have similar benefits. Brouwers et al [26] hypothesized that 3D VR is equivalent to 3D printing in terms of understanding acetabular fracture patterns. They believe that VR can also provide a “realistic” 3D view. The effectiveness of 3D VR and 3D printing in promoting fracture classification was evaluated, and the authors found that 3D VR was less effective than 3D printed models of acetabular fractures. In addition, current 3D VR technology is not suitable for intraoperative use. In the future, advances in VR technology may enable its intraoperative use for the treatment of acetabular fractures.

Iliosacral screw placement is a useful technique for the fixation of posterior pelvic ring injuries. If the pelvic ring is broken, percutaneous iliosacral screw fixation can be performed in the supine position using computer imaging techniques [52-54]. This ensures early fixation for patients with multiple traumas and significantly reduces the risk of bleeding or infectious complications at the surgical site [55]. Tonetti et al [27] aimed to evaluate the educational efficiency of a fluoroscopically guided path simulator for the percutaneous screw fixation of the sacroiliac joint. They evaluated the accuracy of 23 surgeons inserting guidewires according to predetermined procedures in human cadaveric experiments. VR simulation of iliosacral screw insertion was found to reduce the need for intraoperative photography when positioning the guidewire in human cadavers. Novice surgeons who have good anatomical knowledge of the lumbosacral joint but are not used to surgery are the ones who can benefit the most from this valuable tool.

The objective structured assessment of surgical skills provides a method for assessing the technical skills of students [56]. Although this assessment is considered essential, it is rarely conducted because of the high cost, personnel requirements, lack of objectivity of labeling, and possible surgery-related issues [57]. VR has the potential to help overcome some of these issues [58]. Blyth et al [28] recently developed the Bonedoc, a VR simulator for the screw and plate fixation of hip fractures, to solve some of these problems. The Bonedoc simulator integrates all related tasks of hip fracture fixation, from fracture reduction, skin incision, and guidewire placement to final plate-and-screw placement. It automatically calculates accurate positions for fracture reduction and lag screw placement and other objective data. The aforementioned study showed that the Bonedoc simulator could distinguish novice surgeons from surgical trainees; however, its ability to discriminate between basic and advanced trainees was poor. Many studies have used perceptible tactile devices with simulators. Tsai et al [29] introduced a simulator with tactile capabilities to simulate the process of drilling the hip joint during screw and plate surgery and to locate trochanteric hip fractures. Simulation of the drilling process can also be used for surgical training. It is not clear whether a vibration sensation was included in this simulator. Owing to the 1-kHz technical limitation of the response frequency of the Geomagic Touch X5 tactile device, higher-frequency vibrotactile cues may not be accurately replicated by the simulator. In addition, the simulator may not account for the weight of the surgical drill in the trainee’s hands. Surgical training is severely affected by the challenges of reduced training opportunities, shorter working hours, and economic pressure. There is an increasing need to use training systems for training the psychomotor skills of surgical interns. Rambani et al [30] developed a training system for fracture fixation and validated its effectiveness in a cohort of junior orthopedic trainees; see Rambani et al [30] for an image of a computer-assisted orthopedic training system. The computer navigation training system was a good training tool for young orthopedic students. The system could be used to complement the training provided in the operating room. The trainees could be in a threat-free and unhurried environment. The system may be used in other orthopedic operations to learn technical skills and ensure the smooth upgrading of task complexity so as to improve the trainees’ performance of the actual operation in the operating room.

From low-cost task trainers to complex VR solutions, the development of simulators in various surgical specialties has increased substantially. Recent technological advances have enabled the creation of realistic VR environments with tactile feedback. Synthetic bone simulators are the most commonly used simulators in orthopedic training but have considerable limitations: they usually do not simulate soft tissue or use real patient positioning [51]. Racy et al [31] created a VR femoral nail simulator that combines an immersive VR environment with tactile and full image–intensifier functions and then conducted a validation study to evaluate its educational value; see Racy et al [31] for an image of a 3D virtual environment. By integrating multiple aspects of surgical practice into a single device, the authors aimed to improve the participants’ immersion and the tool’s educational value. Thus far, their work has focused on technical skills and shown good authenticity, content, and structural validity.

Given the anticipated increase in the longevity and activity level of patients after total hip arthroplasty, the longevity of the prosthetic components used is critical. Accurate acetabular component positioning is essential to ensure good outcomes. Inaccurate placement may result in impingement, malposition, accelerated wear, loosening of components, and the need for modification. Alexander et al [38] used a radiopaque foam pelvis to simulate component placement; see Alexander et al [38] for an image of component placement using a radiopaque foam pelvis. Cone-beam CT data and optical data from a red-green-blue-depth camera were co-registered to create an AR environment, and the usability of the novel 3D AR guidance system was compared with standard fluoroscopy-guided acetabular component placement. The results showed that the AR technique was more accurate in terms of anteversion and inclination during the placement of the acetabular component than the standard fluoroscopic technique. The AR technique was also faster, without increasing the radiation dose. Similarly, the application of AR technology in acetabular cup placement during total hip arthroplasty was studied by Ogawa et al [39] They developed an acetabular cup placement device, called the AR-hip system, using AR technology. The AR-hip system allows the surgeon to view images of the acetabular cup superimposed on the operative field by means of a smartphone. The smartphone also shows the angle of placement of the acetabular cup. Compared with conventional technology, the AR-hip system provided a more accurate intraoperative acetabular cup placement angle. Although the AR-hip system can display acetabular cup images superimposed on the operating field, the value of this system as a navigation tool is unclear. The time to repeat surgery is influenced by implant wear, which is related to the physical characteristics of the implant as well as the position of the acetabular component. Conversely, appropriate implant placement can restore hip anatomy and biomechanics and reduce the risk of dislocation, impingement, loosening, and limb length discrepancy, thus reducing implant wear and revision rates. An easy-to-use intraoperative component planning system based on 2 C-arm x-ray images combined with 3D AR visualization was presented by Fotouhi et al [40]. This system simplifies the placement of the impactor and acetabular cup by providing real-time red-green-blue-depth data superimposition. The system also helps reduce radiation, operation time, and frustration and increases the efficiency and accuracy of the placement of the acetabular component. Ultimately, this approach may help reduce the rate of revision surgery in patients with hip disease.

For joint replacement, simulation training is typically performed on dry bones or cadavers. The former has low fidelity, whereas the latter has a high cost and requires an internal structure. Neither can objectively measure technology or 3D orientation skills [64]. A MicronTracker camera was integrated with the HoloLens AR headset system by Logishetty et al [41] to develop an enhanced AR headset that can track the position and orientation of the implant relative to the pelvis; see Logishetty et al [41] for an image of an enhanced AR headset. The platform can use real instruments and provide real-time feedback. Therefore, AR is considered a feasible and valuable training tool and can be used as an auxiliary tool for expert guidance in the operating room. Although there was no difference in accuracy between the group trained with AR and the group trained by expert surgeons, the authors believed that the MicronTracker camera and HoloLens AR headset system may be useful in education.

Computer-assisted orthopedic surgery offers obvious advantages for patients, with higher positioning accuracy and fewer outliers; however, its invasiveness, cost, and complexity limit its wide application. To provide seamless computer-aided, improved real-time imaging and a more natural surgical process, Liu et al [42] developed a hip surface replacement navigation system based on AR. Figure 2 shows an AR-based navigation system. To evaluate the accuracy of this navigation system, a pilot hole drilling experiment was conducted using a femoral model. Compared with the preoperative plan, the position and direction of the borehole were found to have an average error of 2 mm and 2°, respectively, and the navigation system was comparable with currently available commercial computer-aided orthopedic systems.

Percutaneous screw osteosynthesis of pelvic fractures performed under conventional imaging guidance represents a challenge for even experienced surgeons [66]. Befrui et al [43] performed K-wire implantation in long bone phantoms and suprapubic phantoms using a red-green-blue-depth augmented cone-beam CT system and compared K-wire placement performed using AR-based navigation with that performed using conventional C-arm fluoroscopy alone. The results showed that AR navigation significantly reduced the operation time—from 9.9 to 4.1 minutes for long bone phantoms and from 10.9 to 5.5 minutes for suprapubic phantoms. Furthermore, AR-based navigation reduced the intraoperative radiation dose. Finally, the placement accuracy did not significantly differ between the conventional method and the AR method.

Percutaneous sacroiliac screw fixation is a widely accepted method for the treatment of posterior pelvic ring instability [67]. Compared with open reduction and internal fixation, percutaneous sacroiliac screw fixation is associated with less trauma and a lower incidence of postoperative wound infection [52]. The conventional method of achieving accurate screw placement involves the insertion of the screws under fluoroscopic guidance. AR can overlay virtual images onto the real world. Wang et al [44] developed a new sacroiliac screw insertion navigation system based on AR for preoperative planning and evaluated its feasibility and accuracy in cadaveric experiments; see Wang et al [44] for an image of a novel AR-based navigation system. Six complete pelvic specimens were imaged using CT scans, and the pelvis and blood vessels were segmented into 3D models. The ideal trajectory of the sacroiliac screw was designed and visualized as a cylinder. For the intervention, a head-mounted display was used to create a real-time AR environment by superimposing a virtual 3D model onto the surgeon’s field of view. According to the trajectory represented by the cylinder, the screw was drilled into the pelvis. This method was feasible and accurate and may be a valuable tool for assisting percutaneous sacroiliac screw implantation in live surgeries.

In the past decades, the application of computer-aided navigation systems in preoperative planning has greatly reduced surgical risks and improved surgical accuracy [68]. Currently, a few commercial surgical navigation systems have been tested and approved, such as ENLight, NavSuite, Portable Nanostation, and MATRIX POLAR. Augmented reality–based surgical navigation system (AR-SNS) is a surgical navigation system based on AR developed by Chen et al [45] that uses an optical transparent head-mounted display; see Chen et al [45] for an image of an optical see-through head-mounted display. The system includes preoperative surgical planning, registration, and intraoperative tracking. With the help of AR-SNS, surgeons wearing a head-mounted display can view merged images that combine virtual anatomical structures such as soft tissues, blood vessels, and nerves with real scenes during surgery so as to improve the safety and reliability of surgery. AR-SNS can be used to implement the preoperative plan. Percutaneous sacroiliac screw implantation is a very common operation in orthopedics. To avoid damaging important anatomical structures such as the soft tissues, blood vessels, and nerves in the pelvis, a virtual path is created for the surgical drill and rendered on all 2D and 3D views, which improves the accuracy, safety, and reliability of implant surgery.

Acetabular fractures remain to be one of the most challenging fractures to treat because of the complex anatomy, difficulty in gaining surgical access to the fracture site, and the relatively low incidence of these lesions, resulting in a long learning curve [46,69] Owing of the rarity and complexity of acetabular fractures, experienced acetabular surgeons are needed to conduct specific teaching and learning tasks [69]. Fornaro et al [46] completed an initial study to test the feasibility of preoperative virtual surgical planning in acetabular fractures using a new prototype planning tool based on an interactive AR environment. Figure 3 shows the feasibility of preoperative surgical planning. The software package Amira (version 3.1, TGS, Inc) was used for semiautomatic segmentation of the pelvic bones and fracture fragments. Then, the segmented images were imported into a planning tool (using OpenGL for graphics and the PHANTOM Omni Developer Kit) in the common Standard Triangle Language or Wavefront Object file formats for haptic rendering. The angle and length of the 3D space were measured according to the specific marks visible or accessible on the pelvic bone during the operation. The pelvic surgery prototype planning tool proposed in this study was successfully integrated into the clinical workflow to improve patient-specific preoperative planning and provide visual and tactile information about the injury. The limitation of this study was that the authors did not use current tools to design and simulate interference by soft tissues. Soft-tissue structures, such as muscles and tendons inserted into the pelvic bones, blood vessels, and pelvic organs, were not modeled.

The conventional surgical treatment method for pelvic and acetabular fractures requires complete exposure of the fractures and intraoperative implant contouring after intraoperative fracture reduction so that the reconstruction plate can adapt to the reduced pelvis. This invasive approach often leads to prolonged operation time and considerable injury and bleeding [70]. Shen et al [47] used a special patient-specific AR-assisted preoperative implant design and unilateral pelvic and acetabular surgical contouring system. This system provides a user-friendly interface for simulating fracture reduction and implant design and a low-cost environment for rapid preoperative implant template development. The entire system consists of 2 subsystems: a virtual fracture reduction system and an AR-based auxiliary template system. The surgeon can design the reconstruction plate and its final shape after bending and create a surgical plan for its placement. This results in the development of a digital preoperative implantation model. The final preoperative reconstruction plate is created by mapping the reconstruction plate kit to the virtual plate kit. The software is implemented in C++ under Windows 7 (Microsoft Corp). In conclusion, by using this type of patient-specific implant template for preoperative surgical planning, the process of intraoperative implant contouring is omitted, which minimizes surgical trauma and enables satisfactory reduction and fixation.

Extracapsular fractures account for a substantial proportion of femoral neck fractures. Extracapsular femoral neck fractures can be treated using a fixed-angle sliding screw device, which is commonly referred to as a sliding compression or dynamic hip screw. However, the mechanical failure rate is as high as 20% [71-73]. To overcome this, van Duren et al [48] developed a digital fluoroscopic imaging simulator using orthogonal cameras to track colored markers attached to guidewires and thereby create a virtual overlay on fluoroscopic images of the hip. This system was used to calculate the virtual guidewire tip-vertex distance and compare it with the physically measured guidewire tip-vertex distance. This study demonstrated a new AR-based simulation of guidewire insertion in dynamic hip screw surgery. Unlike virtual VR, AR can simulate perspective while allowing students to interact with real instruments and perform operations on bone models.

The goal of hip surgery simulation is to improve the operator’s understanding of the anatomy and develop a good touch sensation. Training simulators allow for the repeated practice of surgical techniques before actual surgery, which helps surgeons deepen their visuospatial skills, that is, their visual perception of anatomical structures. Further studies are required to verify whether simulators reduce the odds of poor outcomes and improve the technical ability of trainees to perform actual operations. In addition, more research trials are required to develop simulators for total hip replacement surgery that will shorten the learning curve and increase trainee acceptance. Ideally, the technical skills derived from simulators should be easily transferable to the operating room, thus improving patient satisfaction and helping trainee surgeons accumulate more clinical experience. We believe that the ideal simulator for total hip replacement surgery should be multimodal and provide an immersive environment combining tactile, visual, and auditory cues.

AR technology has successfully provided surgeons with a wide range of visual information about anatomical structures and assisted them throughout the operation. AR technology allows surgeons to view the surgical field through a superimposed 3D virtual model of anatomical details. Dennler et al [95] conducted a clinical feasibility study on AR in the operating room. A total of 13 orthopedic surgeons from a Swiss university clinic performed 25 orthopedic surgical procedures wearing HoloLens, a holographic AR headset providing complementary 3D, patient-specific anatomical information. Although the surgeons were generally satisfied with the image quality of the headset device tested here, they also pointed out some technical and ergonomic deficiencies. However, thus far, only a few studies have evaluated the application of AR in the operating room. This is because in open surgery, the registration of virtual and real scenes remains an open problem. AR registration is affected by problems related to organ deformation, out-of-control breathing, and continuous contact between surgical instruments and soft tissue. The application of AR systems and their components in surgery leads to problems and challenges. AR is an effective, reliable, and promising open surgical technique. However, further improvements are required to improve the performance of AR systems and apply them in different operations. To overcome the problems of organ deformation and inaccurate registration, the virtual model must be updated continuously during the operation.

After hip trauma or surgery, postoperative rehabilitation is essential to restore damaged functions [96]. Successful treatment requires an appropriate exercise combination and progression to improve joint activity and muscle strengthening and restore physical function [96]. The application of VAR in postoperative remote rehabilitation is attracting the interest of orthopedists. In the past decades, remote virtual rehabilitation gained research interest. With the spread of COVID-19, the role of remote virtual rehabilitation has become even more important [97]. Postoperative rehabilitation is widely performed in neurology. van der Veen et al [98] conducted a pilot study quantifying center-of-mass trajectory during dynamic balance tasks using an HTC Vive tracker fixed to the pelvis. HTC Vive can be used to simulate objects, forces, and interactions between objects with high realism and accuracy. Borglund et al [99] studied feedback from HTC Vive sensors and found that the use of this device resulted in transient performance enhancements in a juggling task in VR. Kayabinar et al [100] studied the effects of VAR-based, robot-assisted gait training on dual-task performance and functional measures in patients with chronic stroke. Blasco et al [101] studied the efficacy of VR tools for physical rehabilitation after total knee replacement. The advantages of VR and AR technologies in postoperative remote rehabilitation have been proven in many medical fields [102]; however, few studies have focused on the use of these technologies for orthopedic rehabilitation [103]. Remote rehabilitation has been proven to be safe and effective. During the COVID-19 pandemic, it ensured telemedicine consultations and greatly reduced the risk of unnecessary travel and physical contact. Extending some aspects of medical practice beyond the physical boundaries of clinical medical facilities is a cutting-edge strategy for meeting growing medical needs. It is also necessary to study the cost-effectiveness of telerehabilitation in the future.

We fully believe that VR has made great progress. With AR technology, simulations of alternative environments have been incorporated into rehabilitation therapy. Remote rehabilitation via virtual technology allows high-quality care to be provided at a low cost. In view of the growing demand for orthopedic rehabilitation and the increasing related costs, VAR technology will be increasingly applied in the physical rehabilitation of patients after hip surgery.