Published on in Vol 23, No 9 (2021): September

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/25115, first published .
Mobile Applications in Clinical and Perioperative Care for Anesthesia: Narrative Review

Mobile Applications in Clinical and Perioperative Care for Anesthesia: Narrative Review

Mobile Applications in Clinical and Perioperative Care for Anesthesia: Narrative Review

Authors of this article:

Sabrina Pan 1 Author Orcid Image ;   Lisa Qia Rong 2 Author Orcid Image

Review

1Weill Cornell Medicine, New York, NY, United States

2Department of Anesthesiology, Weill Cornell Medicine - New York-Presbyterian Hospital, New York, NY, United States

Corresponding Author:

Lisa Qia Rong, MD

Department of Anesthesiology

Weill Cornell Medicine - New York-Presbyterian Hospital

525 E 68th St

New York, NY, 10065

United States

Phone: 1 212 746 2962

Email: lir9065@med.cornell.edu


Background: The increasing use of smartphones by providers and patients alike demonstrates that digital health utilizing mobile applications has the potential to transform perioperative care and education in anesthesia.

Objective: This literature review describes the current scope of the use of mobile applications in anesthesiology.

Methods: Literature was searched using PubMed, Scopus, and clinicaltrials.gov for articles published from January 1, 2010, through April 1, 2020. Only English language studies were included. Articles were included if they examined the use of a mobile health application in the setting of anesthesia or the perioperative (immediate preoperative, intraoperative, and postoperative) period. Studies were excluded if they explored video interventions or did not examine the feasibility or efficacy of the mobile app.

Results: We included 29 articles, and three areas of clinical functionality were identified: patient-centered care (preoperative, intraoperative, and postoperative), systems-based improvement, and medical education. Several studies demonstrate the feasibility and reliability of mobile apps in these areas, but many are only tested for efficacy in simulated environments or with small patient samples

Conclusions: Mobile health applications show promise in improving communication between anesthesiologists, improving workflow efficiency, enhancing medical education, and reducing hospital costs. However, there is a need for validation and improvement before full implementation by the provider, patients, and hospital systems. Future studies are needed to demonstrate meaningful health outcomes to create guidelines and recommendations specific to the application of mobile technology to health care.

J Med Internet Res 2021;23(9):e25115

doi:10.2196/25115

Keywords



Mobile health (mHealth)––digital health technologies using mobile phones, tablets, and wearables to improve health outcomes––have the potential to rapidly change the face of medicine, health care, and medical education. By 2019, 81% of Americans owned a smartphone, a significant increase from 35% in 2011 [1]. Additionally, there has been an increased level of interest in using mobile apps to monitor health in the US population, with 45,028 mHealth related applications available for download in Apple’s App Store in 2019, up from 28,343 in 2015 [2]. New advances in mobile apps provide opportunities for innovative patient care, real-time data delivery, and patient-provider engagement in anesthesiology.

Previous reviews in anesthesia have discussed commercially available digital health apps provided in the Apple App Store or Google Play market [3]. This review, however, describes recent clinically tested digital health apps found in scientific literature and how these tools may be integrated into anesthesiologists’ clinical practice.

This review aimed to describe the current scope of mobile application use within anesthesiology. We classified apps found in recent scientific literature into areas of clinical functionality that were deduced from the analysis of the included studies. We also described the limitations of mobile application studies and suggested future directions.


Overview

A review was conducted to answer the following research question: What mHealth applications in anesthesiology have been described in the published literature, and what are their utility and future potential in clinical practice?

Database Search

A complete literature search was conducted with the librarian's assistance using PubMed, Scopus, and clinicaltrials.gov for articles published from January 1, 2010, through April 1, 2020. The search terms were “mobile applications” OR “mobile health” OR “digital health” OR “mobile app” OR “iOS app” OR “mHealth” OR “smartphone” AND “anesthesiology” OR “anesthesia” OR “anesthetic” OR “anesthesiology” OR “anesthesia” OR “anesthetic. The search included titles, abstracts, and keywords for all databases.

Article Selection

Title, abstract, and full text of studies were reviewed by the first author (SP), and the senior author (LR) resolved any ambiguities. Only English language studies were included. Articles were included if they examined the use of an mHealth application in the setting of anesthesia or perioperative evaluation. Studies were excluded if they explored video interventions or did not examine the feasibility or utility of the mobile app.

Data Collection and Summary

Each article was assessed by design, population characteristics, methods, area of clinical functionality, and outcomes. A qualitative analysis was then conducted to summarize the depth and breadth of mHealth apps that have been utilized within anesthesiology. Major clinical functionalities were deduced from this qualitative analysis, and each study was then classified into one of these categories. We then described the limitations of the data from these studies and provided a basis for the future direction of mHealth in the field.


Our search returned 80 studies on PubMed, 141 on Scopus, and 5 studies on clinicaltrials.gov, yielding a total of 226 studies, of which 49 were duplicate studies. The titles and abstracts of the remaining 177 articles were screened. We identified 66 that meet the criteria, and their full text was reviewed, after which 29 articles were selected for a qualitative review.

Within the literature, we identified three areas of clinical functionality: patient-centered care, systems-based improvement, and medical education. Patient-centered care was further subdivided into preoperative, intraoperative, and postoperative monitoring. Certain mobile applications within systems-based improvement were also subcategorized into medication safety or adverse event reporting or guideline compliance. An overview of each study's characteristics, methods, and outcomes can be found in Table 1.

Table 1. Characteristics and outcomes of included studies.
StudyDesignSample, NSample characteristicsCategoryMethodsResults
Cumino et al 2017 [4]RCTa84Pediatric (4-8 years) undergoing elective surgeriesPatient-centered care: PreoperativeFour groups: control (verbal anesthesia), informed (parent given an information leaflet about the anesthetic procedure), smartphone (with a child in holding area), and combined (smartphone or informed). Primary endpoint: m-YPASb in holding area and ORcThe OR anxiety in the control group was higher (76.2%; P=.001) than in the other groups: informed group (38.1%), smartphone group (23.3%), and combined (19.0%).
Lee et al 2013 [5]RCT120Pediatric (1-10 years) surgical patientsPatient-centered care: PreoperativeMidazolam (M group), smartphone app (S), or midazolam & smartphone (SM). Primary outcome: m-YPAS before and after the intervention.Group S score was significantly lower than group M (P=.01, group SM was significantly lower than group M (P<.01), and group SM was significantly lower than group S (P<.01).
Rubin et al 2019 [6]Prospective86Adult surgical patientsPatient-centered care: PreoperativeAn app was designed to administer the Duke Activity Status Index and the 6-minute walk test. Linear regression was performed to estimate the distance walked during the 6MWT from the number of steps measured by the app.Steps measured by app and research-grade pedometer demonstrated intraclass correlation of 0.87 (0.79-0.92; P<.001). Overall model fit was r2=.72 for the distance estimation algorithm.
Barrachina et al 2017 [7]Cross-sectional20Intensive care unit patientsPatient-centered care: IntraoperativeAnalysis of snapshots from patient monitor and photos using Capstesia app were assessed for concordance of PPVd, COe, and dP/dtfIntraclass correlation coefficient or PPV, CO and max dP/dt were 0.991 (95% CI 0.98-0.99), 0.966 (0.96-0.97) and 0.96 (0.95-0.97), respectively.
Carvalho et al 2019 [8]Prospective22Patients with surgery requiring NMBgPatient-centered care: IntraoperativeNMB grade assessed intraoperative care with TOFh ratios obtained by a Stimpod accelerometer versus the new appFor 142 time points, there was no significant difference between the two methods (P=.78). However, insufficient data that the app can diagnose a TOF higher than 0.
Desebbe et al 2016 [9]Prospective2100N/AiPatient-centered care: IntraoperativeThe simulator is used to display arterial waveforms on a computer screen. Data was obtained with different sweep speeds (6 and 12 mm/s) and randomly generated PPV values, pulse pressures, and vitals. Each metric was recorded 5 times at an arterial height scale X1 and 5 times at an arterial height scale X3. Primary outcome: Reproducibility of PPVappj and PPVmank.The precision error of PPVapp and PPVman was 10% (7%-14%) and 6% (3%-10%), respectively. PPVapp shows acceptable accuracy with PPVman when at least three pictures are taken to average PPVapp at scale X1 (upper limit of the 95% CI of the measurement error<12%).
Joosten et al 2019 [10]Cross-sectional40Abdominal surgery patientsPatient-centered care: IntraoperativePPVCAPl compared with SVVPCm at postinduction, preincision, postincision, the end of surgery, and during every hypotensive episode. PPV and SVV are classified into no fluids, gray zone, and fluid administration. Primary outcome: the overall agreement between PPV and SVV and agreement in the fluid administration category549 pairs of PPV-SVV data were obtained. The overall agreement of PPVCAP with SVVPC was 79% ( κ=0.55), demonstrating moderate agreement with only 1% of all measurements resulting in opposite clinical decisions regarding fluid administration.
Joosten et al 2019 [11]Prospective57Elective coronary artery bypass graft patientsPatient-centered care: IntraoperativeThe ability of PPVCAP or PPVPCn to predict fluid responsiveness after infusion of 5 ml/kg of colloid. Primary outcome: overall agreement of PPVCAP and PPVPC as well as COCAPo and CO from COTDpNo difference in the ability of PPVCAP and PPVPC to predict fluid responsiveness (AUROCq 0.74, 95% CI 0.60-0.84 vs AUROC 0.68, 95% CI 0.54-0.80; P =.30). COCAP did not correlate well with COTD.
Chiu et al 2019 [12]RCT156Procedural patients, age 19-75 years,Patient-centered care: PreoperativeCompleted VAS-100r and NRS-11s for pain using paper versus smartphone VAS-100 and NRS-11. Primary outcome: correlation at various time pointsPanda VAS-100 and original tool correlated strongly at emergence (r =.93) and upon discharge (r =.94); Panda NRS-11 correlated strongly with the original at emergence (r =.93) and upon discharge (r =.96)
Dahlberg et al 2017 [13]RCT1027>17 y/o undergoing day surgeryPatient-centered care: PostoperativePatients answered SwQoRt daily for 14 days: smartphone app versus standard care (control). Primary outcome: cost-effectiveness.Net savings of €4.77 (approximately US $5.65) per patient w/ intervention. No difference in SwQoR between the two groups.
Dahlberg et al 2019 [14]RCT494>17 years undergoing day surgeryPatient-centered care: PostoperativePatients randomized to RAPPu daily for 14 days via app versus standard care (no follow-up).62% of contacts made by patients were on postoperative days 1 to 7 and 38% on postoperative days 8 to 14. Demonstrated follow-up should be more long-term.
Highland et al 2019 [15]RCT50Military surgery patientsPatient-centered care: PostoperativeMobile app (mCare) group vs control telephone group to complete Defense and Veterans Pain Rating Scale, nerve block questions, satisfaction surveys, and system usability survey after surgery.No difference in initial response rates between app and control. On day 8, 64% of mCare group completed follow-up versus 48% of the control group (P=.29).
Jaensson et al 2017 [16]RCT997Adult day surgery patientsPatient-centered care: PostoperativeSwQoR daily for 14 days via app intervention (RAPP) or standard care (control). Primary outcome: SwQoR on postoperative days 7 and 14Global SwQoR score was significantly lower (better recovery) in the RAPP group compared to the control group on day 7 (28.23 vs 34.87; P<.001) and day 14 (20.12 vs 21.90; P=.002).
Ke et al 2019 [17]Qualitative study15Obstetric anesthesiologists (9) and cesarean section patients (15)Patient-centered care: PostoperativeStructured phone or in-person interviews with patients and anesthesiologists to assess care after a cesarean section and to solicit feedback on a prototype mobile app for postoperative cesarean section careApp focusing on patient education and self-monitoring. 80% of patients interacted with the app >2x. Most accessed resources were controlling pain, an overview about days after surgery, and key contacts.
Soh et al 2019 [18]RCT42Gastric cancer surgical patientsPatient-centered care: PostoperativeAlarm on the app every 60 min for 2 days with nurse dashboard versus app without alarms or dashboard. Primary outcome: ISIv to assess the frequency of ISI use.ISI was higher in the test group versus control but not significant (113.5 vs 93.2; P=.22). Active coughing showed significantly higher performance in the test group (107.8) compared with the control (94.8).
Sun et al 2014 [19]Prospective62Pediatric patients (4-18 years)Patient-centered care: PostoperativeAssessed pain with both paper FPS-Rw and CASx assessments as well as app-based (Panda) versions postsurgery. Primary outcome: correlation between scores.Panda FPS-R scores correlated strongly with the original tool (r > .93). Panda CAS scores correlated strongly with the original CAS scores at both time points (r>0.87); mean pain scores were higher (up to plus 0.47) with Panda than with the original.
Warren-Stomburg et al 2016 [20]Mixed83Adult surgical patientsPatient-centered care: PostoperativePart I: paper-based questionnaire to identify patients’ attitudes towards follow-up techniques. Part II: feasibility test of a mobile app for follow-up.42.2% prefer to respond by the mobile app in part I, but in part II, adherence to answering questions in the app was only 27.2%. Patients > 60 yrs prefer paper follow-up while patients <40 prefer telemedicine (P=.001).
Baumann et al 2019 [21]Randomized crossover74Resident and attending anesthesiologistsSystems-based: Medication safetyCompared simulations with and without the app. Primary outcome: the probability of administering the accurate dosage.The probability of “accurate” rated dosage was 77.7 (70.9-84.5%) in control versus 94 (90-97.8%) with the app.
Gorges et al 2019 [22]Feasibility studyN/AN/ASystems-basedtelePORT app designed with input from anesthesiology assistants for design features and work-domain analysis. Primary outcome: Usage patterns quantified.telePORT is used more for help requests (approximately 4.5 per day) than team messaging (approximately 1 per day). OR monitoring was frequently utilized (34%). Loss of wireless connectivity was a barrier.
Haffey et al 2013 [23]Retrospective23Opioid conversion appsSystems-based: Medication safetyApps with opioid dose conversion abilities identified. Dose calculations of seven commonly used opioid switches were compared between apps.Of 23 apps, 52% had no stated medical professional involvement. There is a significant difference in mean conversion output for hydromorphone between apps with and without medical professional involvement (0.2256 vs 0.2536; P=.038).
Jabaley et al 2018 [24]Cross-sectional13,846AnesthesiologistsSystems-based: Medication safetyA free mobile app (Anesthesiologist) deployed a 10-question survey about sugammadex use and related adverse events.About 50% of anesthesia providers had access to sugammadex and were given a survey. Anaphylaxis rates are estimated to be between 0.005%-0.098%. 22.7% reported adverse drug reactions.
Lane et al 2012 [25]Feasibility40AnesthesiologistsSystems-basedDevelopment of VigiVU, a mobile app for OR awareness and communication, at Vanderbilt University Medical Center followed by beta testing done with a group of 40.VigiVU push notifications to iPhone were faster than pager (mean 18 sec, SD 8.2). All beta users continued to use the app for the benefits of situational awareness in up to 4 ORs.
Lelaidier et al 2017 [26]RCT52Anesthesia residentsSystems-based: Guideline complianceTwo simulated crises, with and without MAX app. Primary outcome: technical performance during crisis defined by the ESCy. Secondary outcome: nontechnical performanceMean technical performance was higher in the MAX group vs. control group: 81.6%, SD 11.9) versus 58.6%, SD 10.8; P<.001 and nontechnical 33.7%, SD 4.4 versus 30.9%, SD 4.9 points; P<.001.
McEvoy et al 2016 [27]RCT259AnesthesiologistsSystems-based: Guideline complianceA 20-question test regarding clinical scenarios related to ASRAz guidelines using ASRA Coag app versus any resource except the app (control). Primary outcome: test scoreApp group (92.4%, SD 6.6) scored higher than control (68.0%, SD 15.8; P<.001). App use increased the odds of selecting correct answers (7.8, 95% CI 5.7-10.7).
Rothman et al 2013 [28]Cross-sectionalN/AN/ASystems-basedMessage sent with VigiVu versus paging system. Primary outcome: transmission and receipt times calculated as their differences.Mean latencies <1 sec for iPad and iPod devices and <4 sec for iPhone. Service performed better than third party paging systems (Aquis paging system had 0.6% incidence of prolonged message delivery, >100 sec)
Rubin et al 2017 [29]RetrospectiveN/AClinical anesthesia personnelSystems-based: Medication SafetyTwo years before intervention used as the baseline rates of adverse event reporting. Primary outcome: monthly reporting and same-day adverse event reporting compared before and after app implementation.Median reported 12 events for the first year, 14 for the second year, and 20 after the introduction of the mobile app (P=.01). The rate of same-day reporting increased by 10% after the introduction of the app (P=.048).
Burstein et al 2018 [30]FeasibilityN/AN/AMedical educationAn app that to teaches Bier block.It was implemented in 2015 with data collection until 2021.
De Oliveira et al 2013 [31]RCT20Medical studentsMedical education Students were randomized to intervention (iLarynx app for 30 min) versus control (no iLarynx access). Primary outcome: the time required to advance fiberscope from the mouth up to the carina on a mannequin. Failed if carina not seen in <120 sec.80% of the control group failed versus 20% in the intervention group (P=.01). 24 failed attempts in the control group and 4 in the iLarynx group (P<.005).
Linganna et al 2020 [32]RCT18Anesthesiology residentsMedical education Traditional intraoperative teaching of transesophageal echocardiography (control) vs access to EchoEducator app. Primary outcome: score increase from preintervention to postintervention assessment.Intervention group demonstrated a greater increase in score; (plus 19.19%, 95% CI 4.14%-34.24%; P=.02) compared to control.

aRCT: randomized controlled trial.

bm-YPAS: modified Yale Perioperative Anxiety Scale.

cOR: operating room.

dPPV: pulse pressure variation.

eCO: cardiac output.

fdP/dt: max slope of pressure curve.

gNMB: neuromuscular block.

hTOF: train of four.

iN/A: not applicable.

jPPVapp: pulse pressure variation by app.

kPPVman: pulse pressure variation by manual calculation.

lPPVCAP: pulse pressure variation using Capstesia.

mSVVPC: stroke volume variation using pulse contour analysis.

nPPVPC: pulse pressure variation using pulse contour analysis.

oCOCAP: cardiac output using Capstesia.

pCOTD: cardiac output using thermal dilution.

qAUROC: area under the receive operating characteristic

rVAS-100: visual analog scale-100.

sNRS-11: numeric rating scale-11.

tSwQoR: Swedish Quality of Recovery.

uRAPP: recovery assessment by phone points.

vISI: incentive spirometer index.

wFPS-R: faces pain scale-revised.

xCAS: color analog scale.

yESC: European Society for Cardiology.

zASRA: American Society of Regional Anesthesia and Pain Medicine.


Patient-Centered Care

Within anesthesia, the perioperative surgical home has arisen as a model that encompasses the goals of patient-centered care [33]. Mobile apps will play a large role in this area as they can directly engage patients before they arrive at the hospital and after their procedures. By providing on-the-go education, monitoring, and behavioral interventions, the anesthesia team can reduce patient anxiety preoperatively, increase communication through real-time mobile interactions perioperatively, and improve compliance with perioperative instructions.

Preoperative Interventions

Mobile apps provide an opportunity to decrease anxiety without additional sedatives in the pediatric population. Preoperative anxiety in pediatric populations is estimated to be as high as 50% [34]. Lee et al randomized 120 children between ages 1 and 10 (ASA I, II) who were undergoing elective surgery to receive intravenous midazolam (M), behavioral intervention through smartphone applications (eg, Soundtouch interactive, Pororo Sticker Book, Angry Bird) tailored to the child’s developmental status and preferences (S), or both (SM). While they found that anxiety using the modified Yale Preoperative Anxiety Scale (m-YPAS) was lower post-intervention for all three groups (M group: mean 52.8, SD 11.8 vs mean 41.0, SD 7.0; S group: mean 59.2, SD 17.6 vs mean 36.4, SD 7.3; SM group: mean 58.3, SD 17.5 vs mean 26.0, SD 3.4), the S group had lower anxiety levels relative to the M group (P<.01), and the SM group had the lowest level of anxiety (P<.01) [5].

In another randomized controlled trial (RCT) of 84 pediatric patients ages 4 to 8, the children’s median anxiety levels were lower for patients in the smartphone group (parents verbally informed of the anesthetic procedure and child received smartphone app) versus the control group (parent verbally informed about the anesthetic procedure only). The smartphone group had significantly lower m-YPAS scores compared to the control group, measured in the operating room (OR) before anesthesia induction (55.0 vs 23.4; P<.001) [4]. The ability to tailor mobile app choices to a child’s developmental stage demonstrates how mobile technology can be customizable for individual patients.

Another novel use for smartphone apps in the preoperative area is to improve dynamic communications using a centralized platform such as Listeo+. The Listeo+ app provides personalized information to patients, reminders for preoperative recommendations, and a channel for various provider communications. Listeo+ is currently being tested in a multicenter RCT to evaluate the percentage of compliance to preoperative recommendations, thereby decreasing the rate of surgery cancellations [35].

Mobile apps, such as a preoperative functional capacity app, can assist with preoperative risk assessment, even before the patient enters the health care facility. At the University of Chicago, the Step Test app integrates the Duke Activity Status Index (DASI) with the 6-minute walk test (6MWT) [6]. Step Test uses voice prompts and Apple’s CMPedometer to derive calculated step counts for the 6MWT. Rubin et al demonstrated that Step Test’s estimated steps exhibited good agreement with a research-grade pedometer in a cohort of 78 patients (intraclass correlation coefficient of 0.87; P<.001). The app facilitates efficient administration of DASI and 6MWT and provides immediate data to providers without occupying clinic staff. Future applications may include improving functional capacity through patient optimization before procedures. In summary, emerging mHealth apps demonstrate their potential to transform preoperative care by decreasing anxiety and providing coordinated, efficient care. Hopefully, in the future, providing an interface to optimize patient health status before surgery to improve outcomes.

Intraoperative Monitoring

Digital mHealth apps have also been tested as tools to enhance intraoperative monitoring. An ideal mobile app for intraoperative monitoring should be easy-to-use, accurate, noninvasive, cost-effective, and reliable. Capstesia (version 1.1.6; Galenic App SL) is an inexpensive and attractive smartphone app used to calculate pulse pressure variations (PPV) and cardiac output (CO) from a picture taken of the arterial waveform on a monitor screen. In a simulated environment comparing 408 pairs of PPV readings, Capstesia’s PPV demonstrated acceptable accuracy compared with the manual PPV if at least 3 photos of the waveform were taken (measurement error <12%). Accuracy was improved if it was averaged across 5 photos [9] but is less practical in the OR setting. Additional validation studies showed that Capstesia had a percentage of error of 20% for PPV and 13.8% for CO among 20 patients in the intensive care unit [7]. However, when the app was studied in 57 patients undergoing elective cardiac surgeries, its calculated PPV only weakly predicted fluid responsiveness (sensitivity of 73%, 95% CI 0.54-0.92, and a specificity of 74%, 95% CI 0.48-0.90) [11]. When comparing Capstesia’s PPV to stroke volume variation from an uncalibrated pulse wave analysis monitor, Joosten et al found that there was 79% overall agreement between the two, with a kappa coefficient of 0.55 [10]. These results demonstrate the promise and difficulties of creating a mobile app for real-world clinical intraoperative settings.

Digital mHealth apps have also shown promise in intraoperative neuromuscular blockade (NMB) assessment. Often, recovery of NMB is done subjectively due to the lack of accelerators [36,37] and can result in residual weakness and respiratory complications [38]. In a sample of 22 patients, Carvalho et al demonstrated a strong correlation between train-of-four ratios obtained by a standard accelerometer against an Android app used with the phone attached to the patient’s hand (R=0.98). The app also had a small mean difference (0.0004, 95% limits of agreement +/ 0.12) against standard accelerometry [8]. Thus, the digital app is a feasible way to assess NMB. However, further studies are needed to demonstrate efficacy before the widespread application of mobile apps as monitoring devices in the OR.

Postoperative Monitoring

There is an increasing focus on patient-reported outcome measures (PROM), alongside traditional biomarkers, in evaluating clinical care [39]. Mobile apps have emerged as an exciting new tool to track PROMs such as symptoms, pain, and satisfaction with health care delivery beyond the inpatient stay. Apps have been tested in clinical trials demonstrating initial feasibility and cost-effectiveness compared to the standard of care [13,15,16].

One important measure that mobile apps help track is poorly managed acute postoperative pain, which can be related to increased risk of psychological morbidity, decreased quality of life (QoL), and chronic postsurgical pain [40]. The application Pain Assessment using a Novel Digital Application (Panda) contains electronic versions of the faces pain scale revised (FPS-R) and color analog scale (CAS). Figure 1 demonstrates both the original FPS-R and the electronic version seen on the app. In a prospective study of 62 patients, the app version correlated strongly with original scores at various postoperative time points (FPS-R: r>0.93; CAS: r>0.87) [19] and was preferable to the paper version (81% of FPS-R and 76% of CAS participants). Therefore, digital apps may improve adherence to self-reported pain. Additionally, in a study of 156 adult patients undergoing procedures commonly associated with postsurgical pain, Chiu et al found that the app version of the numeric rating scale 11 pain scores were equivalent to that of the paper version when used in adult patients following emergence from anesthesia and at discharge from the postanesthesia care unit [12]. The use of Panda in both adults and pediatric patients demonstrates the app's potential to be used in a variety of patient populations.

Figure 1. Original faces pain scale-revised (top) and adapted electronic faces pain scale-revised for Panda app (bottom) (reproduced with permission from John Wiley and Sons) [19].
View this figure

Another postoperative mobile app was built for patient-initiated inquiry to identify complications earlier and improve patients’ QoL after surgery. An RCT of 1027 day-surgery patients examined postoperative follow-up with a smartphone app known as recovery assessment by phone points (RAPP). The authors found that there were lower health care costs associated with smartphone app follow-up compared to standard of care follow-up (€60.69 vs €37.29; P=.008 with a mean difference of €23.66). However, after the intervention costs were included, net savings were only €4.77 per patient [13]. Patients using the RAPP also reported better global Swedish Quality of Recovery compared to patients using paper assessments on postoperative day 7 (28.23, SD 29.97 vs 34.87, SD 30.68; P<.001) and postoperative day 14 (20.12, SD 26.19 vs 21.90, SD 22.40; P=.002) [16]. The study also helped determine that most patient inquiries (62%) occurred 1 to 7 days postsurgery, most commonly related to the surgical wound (43/119, 36%) and pain (33/119, 28%) [14].

mHealth apps can also incentivize patients to improve respiratory mechanics postoperatively. The Go-breath app was tested in 42 patients who received general anesthesia undergoing a robotic or laparoscopic surgery or laparotomy to encourage the improved use of incentive spirometry (IS). Although it is a nice concept, patients who used the Go-breath app did not score statistically higher on the incentive spirometer index (frequency of IS use over two days) compared to patients who did not (113.5 vs 93.2; P=.22) [18].

Mobile apps can also provide a platform for multidisciplinary care in the postcesarean section period. For example, in a feasibility study including 15 patients of an app designed with obstetric anesthesiologists, 80% of patients used the app at least twice to self-monitor for complications and access pain control resources, thus increasing the amount of patient monitoring for common postcesarean complications and pain [17].

Systems-Based Improvement

For anesthesiologists, smartphone apps are a convenient and accessible way to improve patient safety and quality of care [41,42] through medication safety, adverse event rates, guideline adherence, and responsiveness to emergencies.

For example, immediate response to OR emergencies is of utmost importance, with patient safety and outcomes on the line. Anesthesiologists at Vanderbilt have built an extensive iOS platform known as VigiVu to improve responsiveness through real-time OR videos, vital signs, anesthetic interventions, voice and text communication, and electronic medical record access [25]. Rothman et al demonstrated the superior reliability of the app’s push notifications compared to traditional paging systems—only 0.03% of iPhone notification latencies were over 100 seconds compared to 0.6% for third-party paging latencies [28]. Similarly, anesthesiologists at the University of British Columbia developed an iPhone app, telePORT, to support team-based communication and real-time OR monitoring. Initial results showed that telePORT was successfully integrated for OR help requests (4.5 requests per day) and OR monitoring, representing 34% of app visits [22]. The use of smartphone apps in these situations increases patient safety by prioritizing emergency situations and proper resource allocation. Additionally, data collection from patient monitoring and outcomes in mobile apps can also be utilized for both quality improvement and clinical research in the future.

Medication Safety and Adverse Event Reporting

Mobile apps have also been used to increase adverse event reporting through transparent and convenient processes that help further a culture of safety and quality. An iOS and Android adverse event–reporting app was developed at the University of Chicago (see figure 1 in Rubin et al [29] for an example of the iOS interface used to report events). After implementation, median monthly reporting rates for all providers increased from 12 to 20 (P<.001), with same-day reporting increasing by 10% during the intervention period (P=.048) [29]. Similarly, apps have shown efficacy in monitoring global adverse events regarding the administration of newer drugs. Jabaley et al repurposed a calculator app (Anesthesiologist) to distribute a survey regarding adverse drug reactions associated with sugammadex administration. Using the mobile app, the investigators gathered data from 2770 anesthesia providers that had experience administering the drug across 119 countries [24], demonstrating that an app may be used for crowdsourcing and surveilling new drugs. The app survey found that anaphylaxis rates were estimated to be between 0.005%-0.098%, and 22.7% of survey responders reported witnessing any sugammadex-related adverse drug reactions such as bradycardia and incomplete reversal of NMB [24].

Finally, anesthesiologists have historically used smartphones to calculate drug dosages. Baumann et al compared the probability of administering accurate medication dosages in emergency simulations with a dosage calculator app versus without it [21]. The probability of an accurate dose administration was higher in the app group compared to the control group (94%, SD 90-97.8 vs 77.7%, SD 70.9-84.5) [21]. Apps are often used for medication conversion in the outpatient pain management setting as well. For example, there are many opioid conversion applications designed to reduce medication errors. However, these apps should be used cautiously. In a study examining 23 opioid apps, only 50% provided direct references to sources for their conversion ratios, and over 50% had no documented medical professional involvement [23]. The conversion from 1 mg of oral morphine to oral hydromorphone was statistically different between apps with medical professional involvement versus those without (0.24 vs. 0.25; P=.04), demonstrating that there may be varying degrees of reliability for these mobile apps.

Guideline Compliance

Built-in smartphone-based decision support tools can similarly improve adherence to anesthesia guidelines to reduce technical errors. Numerous reports have found that poor application of guideline standards results in worse patient safety outcomes [43-45]. However, guidelines are numerous, lengthy, and cumbersome to follow. In an RCT across eight institutions, 259 anesthesiologists completed a test requiring the application of the American Society of Regional Anesthesia and Pain Medicine (ASRA) guidelines to clinical situations. The intervention group used an electronic decision support tool, ASRA Coags, which was programmed with the latest guidelines and decision logic, and the control group used any other resource. The authors found that the intervention group had a significantly higher score on a clinical scenario knowledge test (92.4 vs 68.0; P<.001) compared to the control group [27], irrespective of training. Together, these studies show the potential for mobile apps to act as a platform for the rapid application of complex guidelines in clinical situations, potentially reducing adverse events, preventing errors, and improving the quality of care.

Mobile apps have also been developed to improve performance in high-stress situations in lieu of posters, flow charts, and checklists. For example, MAX, a handheld cognitive aid app, was used as an intervention in a RCT of 52 anesthesia residents. Results demonstrated that residents performed better technically, as rated by independent observers when using MAX in simulated crises compared to without MAX use (81.6%, SD 11.9 vs 58.6%, 10.8; P<.001). The app also improved leadership (P=.003), problem-solving (P<.001), and resource-using (P=.006) as assessed by the Ottawa Global Rating scale [26].

Medical Education

The landscape of medical education is changing from the traditional model of classroom teaching to models such as the flipped classroom and asynchronous learning. Mobile technology enables asynchronous learning—a time and location-independent learning model—with greater access to online modules, podcasts, and videos. For example, smartphone apps have now been developed for point-of-care learning of regional anesthetic procedures such as the Bier Block [30]. These apps may provide an all-in-one learning center with videos, cognitive aids, dose calculators, self-timers, and evidence-based references readily available at the learner's convenience [30]. Additionally, education apps use gamification or game-design elements to increase user motivation. Linganna et al developed EchoEducator, a mobile app with transesophageal echocardiography image-based content, and tested it against traditional intraoperative teaching amongst 18 anesthesiology residents over two weeks (please see figure 2 in Linganna et al [32] for examples of questions testing pathology and structures as well as the feedback the resident receives). The app group had a greater knowledge increase than the control group (+19.19%, 95% CI 4.14-34.24; P=.02) in an assessment based on the perioperative transesophageal echocardiography exam [32]. Residents also reported that they would recommend the app to others because of its content customizability and convenience.

Another application of mobile apps in medical education is through simulation. For example, iLarynx utilizes the iPhone’s built-in accelerometer to mimic hand movements for fiberoptic intubation. When tested in 20 novice medical students, 80% of students in the standard training group had at least one failed attempt (>120 seconds) of visualizing the carina compared to 20% of students in the iLarynx group (P=.01). There was also continued group improvement in the iLarynx group but not in the standard training group [31].

Current Obstacles and Future Directions

Despite the many functions that mobile technology could fulfill in anesthesia, various obstacles, including low patient participation, privacy and security, provider responsibility, and shifting workplace norms, ultimately impede their comprehensive integration into the clinical environment. For example, preoperative and postoperative monitoring interventions depend on patient interaction and motivation. While there are some reports of high adherence to mobile app interventions [15], other studies show a poorer response rate, especially among the elderly—a large hurdle in app development [20]. Intraoperative apps need to be assessed for ease of use, reliability, and accuracy in the OR and not distract anesthesiologists. Monitoring apps, although promising, need more clinical validation across a range of populations.

More evidence is needed to demonstrate mobile apps can meaningfully improve clinical outcomes moving forward. Large-scale, prospective studies with outcomes data are necessary to provide sufficient evidence for widespread implementation. Investigators should also consider using the mHealth evidence reporting and assessment guidelines, which address the complex nature of mobile technology research [46].

Limitations

Similar to all narrative reviews, this review is limited by the lack of appraisal criteria for included studies. Additionally, only mobile apps derived from the scientific literature have been included. Thus, there are likely mobile applications apps not studied in the literature that were omitted. We have performed a qualitative review of the search and included only studies that fit into the categories we created, thus excluding other studies.

There are additional limitations to the mHealth apps described. Many are often inadequately tested clinically, lack efficacy data, and do not adhere to standard guidelines. Similarly, there are limited guidelines and protocols on how to use the data collected by these apps, despite their potential for engaging a wide population. Future research should explore if patient engagement through smartphone apps may increase efficiency, patient satisfaction, and outcomes during the perioperative period.

Conclusion

With the rapid uptake of smartphones amongst patients and clinicians alike, mobile applications will likely play a larger role within anesthesiology in the future. mHealth apps have novel roles in improving patient care, efficiency, and intraoperative monitoring during surgery. Mobile apps have also been shown to facilitate systems-wide change by creating a culture of improving patient quality and safety. However, as the field of anesthesiology moves forward into the digital health space, demonstrating feasibility is not sufficient; clinicians must critically evaluate mobile app study protocols and rigor. Despite studies with smaller populations and simulated environments, this review still finds emerging evidence that mHealth applications have the potential to significantly improve communication between anesthesiologists, improve workflow efficiency, enhance medical education, and reduce hospitals’ costs in the perioperative arena.

Conflicts of Interest

None declared.

  1. Mobile Fact Sheet. Pew Research Center. Washington D.C; 2019.   URL: https://www.pewresearch.org/internet/fact-sheet/mobile/ [accessed 2020-06-15]
  2. Appfigures. Number of mHealth apps available in the Apple App Store from 1st quarter 2015 to 1st quarter.: Statista   URL: https://www.statista.com/statistics/779910/health-apps-available-ios-worldwide/ [accessed 2020-06-15]
  3. Rothman BS, Gupta RK, McEvoy MD. Mobile Technology in the Perioperative Arena. Anesthesia & Analgesia 2017;124(3):807-818. [CrossRef]
  4. Cumino D, Vieira J, Lima L, Stievano L, Silva R. Mathias LAST. Smartphone-based behavioural intervention alleviates children's anxiety during anaesthesia induction: A randomised controlled trial. European Journal of Anaesthesiology (EJA) 2017;34(3):A. [CrossRef]
  5. Lee J, Jung H, Lee G, Kim H, Park S, Woo S. Effect of behavioral intervention using smartphone application for preoperative anxiety in pediatric patients. Korean J Anesthesiol 2013 Dec;65(6):508-518 [FREE Full text] [CrossRef] [Medline]
  6. Rubin D, Dalton A, Tank A, Berkowitz M, Arnolds D, Liao C, et al. Development and Pilot Study of an iOS Smartphone Application for Perioperative Functional Capacity Assessment. Anesth Analg 2020 Sep;131(3):830-839. [CrossRef] [Medline]
  7. Barrachina B, Cobos RFMN, Mardones NFCA, Castañeda A, Vinuesa C. Assessment of a smartphone app (Capstesia) for measuring pulse pressure variation: agreement between two methods: A Cross-sectional study. Eur J Anaesthesiol 2017 Feb;34(2):75-80. [CrossRef] [Medline]
  8. Carvalho H, Verdonck M, Berghmans J, Poelaert J. Development and validation of an android-based application for anaesthesia neuromuscular monitoring. J Clin Monit Comput 2019 Oct 16;33(5):863-870. [CrossRef] [Medline]
  9. Desebbe O, Joosten A, Suehiro K, Lahham S, Essiet M, Rinehart J, et al. A Novel Mobile Phone Application for Pulse Pressure Variation Monitoring Based on Feature Extraction Technology. Anesthesia & Analgesia 2016;123(1):105-113. [CrossRef]
  10. Joosten A, Jacobs A, Desebbe O, Vincent J, Sarah S, Rinehart J, et al. Monitoring of pulse pressure variation using a new smartphone application (Capstesia) versus stroke volume variation using an uncalibrated pulse wave analysis monitor: a clinical decision making study during major abdominal surgery. J Clin Monit Comput 2019 Oct 03;33(5):787-793. [CrossRef] [Medline]
  11. Joosten A, Boudart C, Vincent J, Vanden Eynden F, Barvais L, Van Obbergh L, et al. Ability of a New Smartphone Pulse Pressure Variation and Cardiac Output Application to Predict Fluid Responsiveness in Patients Undergoing Cardiac Surgery. Anesthesia & Analgesia 2019;128(6):1145-1151. [CrossRef]
  12. Chiu LYL, Sun T, Ree R, Dunsmuir D, Dotto A, Ansermino JM, et al. The evaluation of smartphone versions of the visual analogue scale and numeric rating scale as postoperative pain assessment tools: a prospective randomized trial. Can J Anaesth 2019 Jun 22;66(6):706-715. [CrossRef] [Medline]
  13. Dahlberg K, Philipsson A, Hagberg L, Jaensson M, Hälleberg-Nyman M, Nilsson U. Cost-effectiveness of a systematic e-assessed follow-up of postoperative recovery after day surgery: a multicentre randomized trial. Br J Anaesth 2017 Nov 01;119(5):1039-1046 [FREE Full text] [CrossRef] [Medline]
  14. Dahlberg K, Jaensson M, Nilsson U. "Let the patient decide" - Person-centered postoperative follow-up contacts, initiated via a phone app after day surgery: Secondary analysis of a randomized controlled trial. Int J Surg 2019 Jan;61:33-37 [FREE Full text] [CrossRef] [Medline]
  15. Highland K, Tran J, Edwards H, Bedocs P, Suen J, Buckenmaier C. Feasibility of App-Based Postsurgical Assessment of Pain, Pain Impact, and Regional Anesthesia Effects: A Pilot Randomized Controlled Trial. Pain Med 2019 Aug 01;20(8):1592-1599. [CrossRef] [Medline]
  16. Jaensson M, Dahlberg K, Eriksson M, Nilsson U. Evaluation of postoperative recovery in day surgery patients using a mobile phone application: a multicentre randomized trial. Br J Anaesth 2017 Nov 01;119(5):1030-1038 [FREE Full text] [CrossRef] [Medline]
  17. Ke JXC, George RB, Wozney L, Chorney JL. Patient-centred perioperative mobile application in Cesarean delivery: needs assessment and development. Can J Anaesth 2019 Oct 13;66(10):1194-1201. [CrossRef] [Medline]
  18. Soh JY, Lee SU, Lee I, Yoon KS, Song C, Kim NH, et al. A Mobile Phone–Based Self-Monitoring Tool for Perioperative Gastric Cancer Patients With Incentive Spirometer: Randomized Controlled Trial. JMIR Mhealth Uhealth 2019 Feb 19;7(2):e12204. [CrossRef]
  19. Sun T, West N, Ansermino JM, Montgomery CJ, Myers D, Dunsmuir D, et al. A smartphone version of the Faces Pain Scale-Revised and the Color Analog Scale for postoperative pain assessment in children. Paediatr Anaesth 2015 Dec 28;25(12):1264-1273. [CrossRef] [Medline]
  20. Warren-Stomberg M, Jacobsson J, Brattwall M, Jildenstål P. At-home monitoring after surgery/anaesthesia - a challenge. J Eval Clin Pract 2016 Dec 01;22(6):882-886. [CrossRef] [Medline]
  21. Baumann D, Dibbern N, Sehner S, Zöllner C, Reip W, Kubitz JC. Validation of a mobile app for reducing errors of administration of medications in an emergency. J Clin Monit Comput 2019 Jun 2;33(3):531-539. [CrossRef] [Medline]
  22. Görges M, West NC, Petersen CL, Ansermino JM. Development and Implementation of the Portable Operating Room Tracker App With Vital Signs Streaming Infrastructure: Operational Feasibility Study. JMIR Perioper Med 2019 Aug 05;2(2):e13559 [FREE Full text] [CrossRef] [Medline]
  23. Haffey F, Brady RRW, Maxwell S. A comparison of the reliability of smartphone apps for opioid conversion. Drug Saf 2013 Feb;36(2):111-117. [CrossRef] [Medline]
  24. Jabaley CS, Wolf FA, Lynde GC, O'Reilly-Shah VN. Crowdsourcing sugammadex adverse event rates using an in-app survey: feasibility assessment from an observational study. Ther Adv Drug Saf 2018 Jul;9(7):331-342 [FREE Full text] [CrossRef] [Medline]
  25. Lane JS, Sandberg WS, Rothman B. Development and implementation of an integrated mobile situational awareness iPhone application VigiVU™ at an academic medical center. Int J Comput Assist Radiol Surg 2012 Sep 7;7(5):721-735. [CrossRef] [Medline]
  26. Lelaidier R, Balança B, Boet S, Faure A, Lilot M, Lecomte F, et al. Use of a hand-held digital cognitive aid in simulated crises: the MAX randomized controlled trial. Br J Anaesth 2017 Nov 01;119(5):1015-1021 [FREE Full text] [CrossRef] [Medline]
  27. McEvoy M, Hand WR, Stiegler MP, DiLorenzo AN, Ehrenfeld JM, Moran KR, et al. A Smartphone-based Decision Support Tool Improves Test Performance Concerning Application of the Guidelines for Managing Regional Anesthesia in the Patient Receiving Antithrombotic or Thrombolytic Therapy. Anesthesiology 2016 Jan;124(1):186-198 [FREE Full text] [CrossRef] [Medline]
  28. Rothman BS, Dexter F, Epstein RH. Communication Latencies of Apple Push Notification Messages Relevant for Delivery of Time-Critical Information to Anesthesia Providers. Anesthesia & Analgesia 2013;117(2):398-404. [CrossRef]
  29. Rubin DS, Pesyna C, Jakubczyk S, Liao C, Tung A. Introduction of a Mobile Adverse Event Reporting System Is Associated With Participation in Adverse Event Reporting. Am J Med Qual 2019 Jun 25;34(1):30-35. [CrossRef] [Medline]
  30. Burstein B, Bretholz A. A Novel Smartphone App to Support Learning and Maintaining Competency With Bier Blocks for Pediatric Forearm Fracture Reductions: Protocol for a Mixed-Methods Study. JMIR Res Protoc 2018 Dec 21;7(12):e10363 [FREE Full text] [CrossRef] [Medline]
  31. De Oliveira GS, Glassenberg R, Chang R, Fitzgerald P, McCarthy RJ. Virtual airway simulation to improve dexterity among novices performing fibreoptic intubation. Anaesthesia 2013 Oct;68(10):1053-1058 [FREE Full text] [CrossRef] [Medline]
  32. Linganna RE, Patel SJ, Ghofaily LA, Mackay EJ, Spelde AE, Zhou EY, et al. Pilot Study Suggests Smartphone Application Knowledge Improves Resident Transesophageal Echocardiography Knowledge: A Randomized Controlled Trial. J Cardiothorac Vasc Anesth 2020 Aug;34(8):2126-2132. [CrossRef] [Medline]
  33. Kain ZN, Vakharia S, Garson L, Engwall S, Schwarzkopf R, Gupta R, et al. The Perioperative Surgical Home as a Future Perioperative Practice Model. Anesthesia & Analgesia 2014;118(5):1126-1130. [CrossRef]
  34. Davidson AJ, Shrivastava PP, Jamsen K, Huang GH, Czarnecki C, Gibson MA, et al. Risk factors for anxiety at induction of anesthesia in children: a prospective cohort study. Paediatr Anaesth 2006 Sep;16(9):919-927. [CrossRef] [Medline]
  35. Herrera-Usagre M, Santana V, Burgos-Pol R, Oliva JP, Sabater E, Rita-Acosta M, et al. Effect of a Mobile App on Preoperative Patient Preparation for Major Ambulatory Surgery: Protocol for a Randomized Controlled Trial. JMIR Res Protoc 2019 Jan 16;8(1):e10938 [FREE Full text] [CrossRef] [Medline]
  36. Naguib M, Kopman AF, Lien CA, Hunter JM, Lopez A, Brull SJ. A Survey of Current Management of Neuromuscular Block in the United States and Europe. Anesthesia & Analgesia 2010;111(1):110-119. [CrossRef]
  37. Söderström CM, Eskildsen KZ, Gätke MR, Staehr-Rye AK. Objective neuromuscular monitoring of neuromuscular blockade in Denmark: an online-based survey of current practice. Acta Anaesthesiol Scand 2017 Jul 02;61(6):619-626. [CrossRef] [Medline]
  38. Bulka C, Terekhov MA, Martin B, Dmochowski R, Hayes R, Ehrenfeld J. Nondepolarizing Neuromuscular Blocking Agents, Reversal, and Risk of Postoperative Pneumonia. Anesthesiology 2016 Oct;125(4):647-655 [FREE Full text] [CrossRef] [Medline]
  39. Weldring T, Smith SMS. Patient-Reported Outcomes (PROs) and Patient-Reported Outcome Measures (PROMs). Health Serv Insights 2013;6:61-68 [FREE Full text] [CrossRef] [Medline]
  40. Shipton EA. The Transition from Acute to Chronic Post Surgical Pain. Anaesth Intensive Care 2011 Sep 01;39(5):824-836. [CrossRef]
  41. Ozdalga E, Ozdalga A, Ahuja N. The smartphone in medicine: a review of current and potential use among physicians and students. J Med Internet Res 2012 Sep 27;14(5):e128 [FREE Full text] [CrossRef] [Medline]
  42. Nabors C, Peterson SJ, Aronow W, Sule S, Mumtaz A, Shah T, et al. Mobile physician reporting of clinically significant events-a novel way to improve handoff communication and supervision of resident on call activities. J Patient Saf 2014 Dec;10(4):211-217. [CrossRef] [Medline]
  43. McEvoy MD, Field LC, Moore HE, Smalley JC, Nietert PJ, Scarbrough SH. The effect of adherence to ACLS protocols on survival of event in the setting of in-hospital cardiac arrest. Resuscitation 2014 Jan;85(1):82-87 [FREE Full text] [CrossRef] [Medline]
  44. Affronti ML, Schneider SM, Herndon JE, Schlundt S, Friedman HS. Adherence to antiemetic guidelines in patients with malignant glioma: a quality improvement project to translate evidence into practice. Support Care Cancer 2014 Jul 26;22(7):1897-1905 [FREE Full text] [CrossRef] [Medline]
  45. Mearis M, Shega JW, Knoebel RW. Does Adherence to National Comprehensive Cancer Network Guidelines Improve Pain-Related Outcomes? An Evaluation of Inpatient Cancer Pain Management at an Academic Medical Center. Journal of Pain and Symptom Management 2014 Sep;48(3):451-458. [CrossRef]
  46. Agarwal S, LeFevre AE, Lee J, L'Engle K, Mehl G, Sinha C, WHO mHealth Technical Evidence Review Group. Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklist. BMJ 2016 Mar 17;352:i1174. [CrossRef] [Medline]


6MWT: 6 Minute Walk Test
ASRA: American Society of Regional Anesthesia
CAS: color analog scale
CO: cardiac output
DASI: Duke Activity Status Index
FPS-R: faces pain scale-revised
IS: incentive spirometry
ICC: intraclass correlation coefficients
mHealth: mobile health
mYPAS: modified Yale Preoperative Anxiety Scale
NMB: neuromuscular blockade
OR: operating room
PPV: pulse pressure variation
PROM: patient-reported outcome measures
QoL: quality of life
RAPP: recovery assessment by phone points
RCT: randomized controlled trial


Edited by R Kukafka; submitted 22.10.20; peer-reviewed by AS Poncette, C Reis; comments to author 08.12.20; revised version received 28.12.20; accepted 14.06.21; published 17.09.21

Copyright

©Sabrina Pan, Lisa Qia Rong. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 17.09.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.