Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Journal Description

The Journal of Medical Internet Research (JMIR), now in its 21st year, is the pioneer open access eHealth journal and is the flagship journal of JMIR Publications. It is the leading digital health journal globally in terms of quality/visibility (Impact Factor 2019: 5.03), ranking Q1 in the medical informatics category, and is also the largest journal in the field. The journal focuses on emerging technologies, medical devices, apps, engineering, telehealth and informatics applications for patient education, prevention, population health and clinical care. As a leading high-impact journal in its disciplines (health informatics and health services research), it is selective, but it is now complemented by almost 30 specialty JMIR sister journals, which have a broader scope, and which together receive over 6.000 submissions a year. Peer-review reports are portable across JMIR journals and papers can be transferred, so authors save time by not having to resubmit a paper to different journal but can simply transfer it between journals. 

As an open access journal, we are read by clinicians, allied health professionals, informal caregivers, and patients alike, and have (as with all JMIR journals) a focus on readable and applied science reporting the design and evaluation of health innovations and emerging technologies. We publish original research, viewpoints, and reviews (both literature reviews and medical device/technology/app reviews).

We are also a leader in participatory and open science approaches, and offer the option to publish new submissions immediately as preprints, which receive DOIs for immediate citation (eg, in grant proposals), and for open peer-review purposes. We also invite patients to participate (eg, as peer-reviewers) and have patient representatives on editorial boards.

Be a widely cited leader in the digitial health revolution and submit your paper today!


Recent Articles:

  • Source: freepik; Copyright: jcomp; URL:; License: Licensed by JMIR.

    Minimizing the Impact of the COVID-19 Epidemic on Oncology Clinical Trials: Retrospective Study of Beijing Cancer Hospital


    Background: In view of repeated COVID-19 outbreaks in most countries, clinical trials will continue to be conducted under outbreak prevention and control measures for the next few years. It is very significant to explore an optimal clinical trial management model during the outbreak period to provide reference and insight for other clinical trial centers worldwide. Objective: The aim of this study was to explore the management strategies used to minimize the impact of the COVID-19 epidemic on oncology clinical trials. Methods: We implemented a remote management model to maintain clinical trials conducted at Beijing Cancer Hospital, which realized remote project approval, remote initiation, remote visits, remote administration and remote monitoring to get through two COVID-19 outbreaks in the capital city from February to April and June to July 2020. The effectiveness of measures was evaluated as differences in rates of protocol compliance, participants lost to follow-up, participant withdrawal, disease progression, participant mortality, and detection of monitoring problems. Results: During the late of the first outbreak, modifications were made in trial processing, participant management and quality control, which allowed the hospital to ensure the smooth conduct of 572 trials, with a protocol compliance rate of 85.24% for 3718 participants across both outbreaks. No COVID-19 infections were recorded among participants or trial staff, and no major procedural errors occurred between February and July 2020. These measures led to significantly higher rates of protocol compliance and significantly lower rates of loss to follow-up or withdrawal after the second outbreak than after the first, without affecting rates of disease progression or mortality. The hospital provided trial sponsors with a remote monitoring system in a timely manner, and 3820 trial issues were identified. Conclusions: When public health emergencies occur, an optimal clinical trial model combining on-site and remote management could guarantee the health care and treatment needs of clinical trial participants, in which remote management plays a key role.

  • Source: Image created by the Author/Adobe Stock; Copyright: The Authors/tippapatt; URL:; License: Licensed by the authors.

    What Every Reader Should Know About Studies Using Electronic Health Record Data but May Be Afraid to Ask


    Coincident with the tsunami of COVID-19–related publications, there has been a surge of studies using real-world data, including those obtained from the electronic health record (EHR). Unfortunately, several of these high-profile publications were retracted because of concerns regarding the soundness and quality of the studies and the EHR data they purported to analyze. These retractions highlight that although a small community of EHR informatics experts can readily identify strengths and flaws in EHR-derived studies, many medical editorial teams and otherwise sophisticated medical readers lack the framework to fully critically appraise these studies. In addition, conventional statistical analyses cannot overcome the need for an understanding of the opportunities and limitations of EHR-derived studies. We distill here from the broader informatics literature six key considerations that are crucial for appraising studies utilizing EHR data: data completeness, data collection and handling (eg, transformation), data type (ie, codified, textual), robustness of methods against EHR variability (within and across institutions, countries, and time), transparency of data and analytic code, and the multidisciplinary approach. These considerations will inform researchers, clinicians, and other stakeholders as to the recommended best practices in reviewing manuscripts, grants, and other outputs from EHR-data derived studies, and thereby promote and foster rigor, quality, and reliability of this rapidly growing field.

  • Source: iStock; Copyright: Andrey Popov; URL:; License: Licensed by the authors.

    Preferences for Artificial Intelligence Clinicians Before and During the COVID-19 Pandemic: Discrete Choice Experiment and Propensity Score Matching Study


    Background: Artificial intelligence (AI) methods can potentially be used to relieve the pressure that the COVID-19 pandemic has exerted on public health. In cases of medical resource shortages caused by the pandemic, changes in people’s preferences for AI clinicians and traditional clinicians are worth exploring. Objective: We aimed to quantify and compare people’s preferences for AI clinicians and traditional clinicians before and during the COVID-19 pandemic, and to assess whether people’s preferences were affected by the pressure of pandemic. Methods: We used the propensity score matching method to match two different groups of respondents with similar demographic characteristics. Respondents were recruited in 2017 and 2020. A total of 2048 respondents (2017: n=1520; 2020: n=528) completed the questionnaire and were included in the analysis. Multinomial logit models and latent class models were used to assess people’s preferences for different diagnosis methods. Results: In total, 84.7% (1115/1317) of respondents in the 2017 group and 91.3% (482/528) of respondents in the 2020 group were confident that AI diagnosis methods would outperform human clinician diagnosis methods in the future. Both groups of matched respondents believed that the most important attribute of diagnosis was accuracy, and they preferred to receive combined diagnoses from both AI and human clinicians (2017: odds ratio [OR] 1.645, 95% CI 1.535-1.763; P<.001; 2020: OR 1.513, 95% CI 1.413-1.621; P<.001; reference: clinician diagnoses). The latent class model identified three classes with different attribute priorities. In class 1, preferences for combined diagnoses and accuracy remained constant in 2017 and 2020, and high accuracy (eg, 100% accuracy in 2017: OR 1.357, 95% CI 1.164-1.581) was preferred. In class 2, the matched data from 2017 were similar to those from 2020; combined diagnoses from both AI and human clinicians (2017: OR 1.204, 95% CI 1.039-1.394; P=.011; 2020: OR 2.009, 95% CI 1.826-2.211; P<.001; reference: clinician diagnoses) and an outpatient waiting time of 20 minutes (2017: OR 1.349, 95% CI 1.065-1.708; P<.001; 2020: OR 1.488, 95% CI 1.287-1.721; P<.001; reference: 0 minutes) were consistently preferred. In class 3, the respondents in the 2017 and 2020 groups preferred different diagnosis methods; respondents in the 2017 group preferred clinician diagnoses, whereas respondents in the 2020 group preferred AI diagnoses. In the latent class, which was stratified according to sex, all male and female respondents in the 2017 and 2020 groups believed that accuracy was the most important attribute of diagnosis. Conclusions: Individuals’ preferences for receiving clinical diagnoses from AI and human clinicians were generally unaffected by the pandemic. Respondents believed that accuracy and expense were the most important attributes of diagnosis. These findings can be used to guide policies that are relevant to the development of AI-based health care.

  • Source: Rawpixel; Copyright:; URL:; License: Licensed by JMIR.

    Barriers to the Large-Scale Adoption of a COVID-19 Contact Tracing App in Germany: Survey Study


    Background: During the COVID-19 pandemic, one way to reduce further transmissions of SARS-CoV-2 is the widespread use of contact tracing apps. Such apps keep track of proximity contacts and warn contacts of persons who tested positive for an infection. Objective: In this study, we analyzed potential barriers to the large-scale adoption of the official contact tracing app that was introduced in Germany on June 16, 2020. Methods: Survey data were collected from 3276 adults during the week the app was introduced using an offline-recruited, probability-based online panel of the general adult population in Germany. Results: We estimate that 81% of the population aged 18 to 77 years possess the devices and ability to install the official app and that 35% are also willing to install and use it. Potential spreaders show high access to devices required to install the app (92%) and high ability to install the app (91%) but low willingness (31%) to correctly adopt the app, whereas for vulnerable groups, the main barrier is access (62%). Conclusions: The findings suggest a pessimistic view on the effectiveness of app-based contact tracing to contain the COVID-19 pandemic. We recommend targeting information campaigns at groups with a high potential to spread the virus but who are unwilling to install and correctly use the app, in particular men and those aged between 30 and 59 years. In addition, vulnerable groups, in particular older individuals and those in lower-income households, may be provided with equipment and support to overcome their barriers to app adoption.

  • Source: Unsplash; Copyright: Chelsea Gates; URL:; License: Licensed by JMIR.

    Development of a Web-Based Mindfulness Program for People With Multiple Sclerosis: Qualitative Co-Design Study


    Background: Mindfulness-based stress reduction is an efficacious treatment for people with chronic health problems; however, it is highly intensive and time-consuming, which is a barrier for service provision. Objective: This study aims to develop an internet-delivered adapted version of mindfulness-based stress reduction for people with multiple sclerosis to make the intervention more accessible. Methods: We co-designed a web-based mindfulness program with end users, that is, people with multiple sclerosis (N=19). Iterative feedback was also collected from a subsample of the initial group of end users (n=11), and the program was reviewed by experts (n=8). Results: We identified three main themes common to people with multiple sclerosis: dealing with uncertainty and fears for the future, grief and loss, and social isolation. These themes were incorporated into narratives throughout the program. People with multiple sclerosis who reviewed the program gave feedback that the program was relatable, feasible, and acceptable. Experts agreed that the program appropriately represented the main tenets of mindfulness. Iterative feedback was used to further refine the program. Conclusions: The web-based mindfulness program that we developed was viewed positively by both experts and end users. The program reflects common concerns for people with multiple sclerosis and has the potential to meet important unmet psychological needs. A randomized controlled trial was planned to determine the efficacy of the program.

  • Smart glass based skill training for undergraduate nursing students. Source: Image created by the authors; Copyright: The Authors; URL:; License: Creative Commons Attribution + ShareAlike (CC-BY-SA).

    Adaptation of Extended Reality Smart Glasses for Core Nursing Skill Training Among Undergraduate Nursing Students: Usability and Feasibility Study


    Background: Skill training in nursing education has been highly dependent on self-training because of Korea’s high student-faculty ratio. Students tend to have a passive attitude in self-practice, and it is hard to expect effective learning outcomes with traditional checklist-dependent self-practice. Smart glasses have a high potential to assist nursing students with timely information, and a hands-free device does not interrupt performance. Objective: This study aimed to develop a smart glass–based nursing skill training program and evaluate its usability and feasibility for the implementation of self-practice. Methods: We conducted a usability and feasibility study with 30 undergraduate nursing students during a 2-hour open lab for self-practice of core nursing skills, wearing smart glasses for visualized guidance. The usability test was conducted using a 16-item self-reporting questionnaire and 7 open-ended questions. Learning satisfaction was assessed using a 7-item questionnaire. The number of practice sessions was recorded, and perceived competency in core nursing skills was measured before and after the intervention. At the final evaluation, performance accuracy and time consumed for completion were recorded. Results: Smart glass–assisted self-practice of nursing skills was perceived as helpful, convenient, and interesting. Participants reported improved recollection of sequences of skills, and perceived competency was significantly improved. Several issues were raised by participants regarding smart glasses, including small screen size, touch sensors, fogged lenses with masks, heaviness, and heat after a period of time. Conclusions: Smart glasses have the potential to assist self-practice, providing timely information at students’ own paces. Having both hands free from holding a device, participants reported the convenience of learning as they could practice and view the information simultaneously. Further revision correcting reported issues would improve the applicability of smart glasses in other areas of nursing education.

  • Source: Placeit / Heali AI; Copyright: Placeit / Heali AI; URL:; License: Licensed by JMIR.

    A Novel Mobile App (Heali) for Disease Treatment in Participants With Irritable Bowel Syndrome: Randomized Controlled Pilot Trial


    Background: A diet high in fermentable, oligo-, di-, monosaccharides and polyols (FODMAPs) has been shown to exacerbate symptoms of irritable bowel syndrome (IBS). Previous literature reports significant improvement in IBS symptoms with initiation of a low FODMAP diet (LFD) and monitored reintroduction. However, dietary adherence to the LFD is difficult, with patients stating that the information given by health care providers is often generalized and nonspecific, requiring them to search for supplementary information to fit their needs. Objective: The aim of our study was to determine whether Heali, a novel artificial intelligence dietary mobile app can improve adherence to the LFD, IBS symptom severity, and quality of life outcomes in adults with IBS or IBS-like symptoms over a 4-week period. Methods: Participants were randomized into 2 groups: the control group (CON), in which participants received educational materials, and the experimental group (APP), in which participants received access to the mobile app and educational materials. Over the course of this unblinded online trial, all participants completed a battery of 5 questionnaires at baseline and at the end of the trial to document IBS symptoms, quality of life, LFD knowledge, and LFD adherence. Results: We enrolled 58 participants in the study (29 in each group), and 25 participants completed the study in its entirety (11 and 14 for the CON and APP groups, respectively). Final, per-protocol analyses showed greater improvement in quality of life score for the APP group compared to the CON group (31.1 and 11.8, respectively; P=.04). Reduction in total IBS symptom severity score was 24% greater for the APP group versus the CON group. Although this did not achieve significance (–170 vs –138 respectively; P=.37), the reduction in the subscore for bowel habit dissatisfaction was 2-fold greater for the APP group than for the CON group (P=.05). Conclusions: This initial study provides preliminary evidence that Heali may provide therapeutic benefit to its users, specifically improvements in quality of life and bowel habits. Although this study was underpowered, findings from this study warrant further research in a larger sample of participants to test the efficacy of Heali app use to improve outcomes for patients with IBS. Trial Registration: NCT04256551;

  • Source: pexels; Copyright: C Technical; URL:; License: Licensed by JMIR.

    The Effect of Information and Communication Technology and Social Networking Site Use on Older People’s Well-Being in Relation to Loneliness: Review of...


    Background: In the last decades, the relationship between social networking sites (SNSs) and older people’s loneliness is gaining specific relevance. Studies in this field are often based on qualitative methods to study in-depth self-perceived issues, including loneliness and well-being, or quantitative surveys to report the links between information and communication technologies (ICTs) and older people’s well-being or loneliness. However, these nonexperimental methods are unable to deeply analyze the causal relationship. Moreover, the research on older people’s SNS use is still scant, especially regarding its impact on health and well-being. In recent years, the existing review studies have separately focused their attention on loneliness and social isolation of older people or on the use of ICTs and SNSs in elderly populations without addressing the relationship between the former and the latter. This thorough qualitative review provides an analysis of research performed using an experimental or quasi-experimental design that investigates the causal effect of ICT and SNS use on elderly people’s well-being related to loneliness. Objective: The aims of this review are to contrast and compare research designs (sampling and recruitment, evaluation tools, interventions) and the findings of these studies and highlight their limitations. Methods: Using an approach that integrates the methodological framework for scoping studies and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for systematic reviews, we identified 11 articles that met our inclusion criteria. A thematic and content analysis was performed based on the ex post categorization of the data on the selected studies, and the data were summarized in tables. Results: The analysis of the selected articles showed that: (1) ICT use is positively but weakly related to the different measures of older people’s well-being and loneliness, (2) overall, the studies under review lack a sound experimental design, (3) the main limitations of these studies lie in the lack of rigor in the sampling method and in the recruitment strategy. Conclusions: The analysis of the reviewed studies confirms the existence of a beneficial effect of ICT use on the well-being of older people in terms of reduced loneliness. However, the causal relationship is often found to be weak. This review highlights the need to study these issues further with adequate methodological rigor.

  • COVID Coach home screen featured on an iPhone which is laying on a table in an outdoor setting. Source: The authors / Placeit; Copyright: The authors / Placeit; URL:; License: Licensed by JMIR.

    Exploring Usage of COVID Coach, a Public Mental Health App Designed for the COVID-19 Pandemic: Evaluation of Analytics Data


    Background: The COVID-19 pandemic has significantly impacted mental health and well-being. Mobile mental health apps can be scalable and useful tools in large-scale disaster responses and are particularly promising for reaching vulnerable populations. COVID Coach is a free, evidence-informed mobile app designed specifically to provide tools and resources for addressing COVID-19–related stress. Objective: The purpose of this study was to characterize the overall usage of COVID Coach, explore retention and return usage, and assess whether the app was reaching individuals who may benefit from mental health resources. Methods: Anonymous usage data collected from COVID Coach between May 1, 2020, through October 31, 2020, were extracted and analyzed for this study. The sample included 49,287 unique user codes and 3,368,931 in-app events. Results: Usage of interactive tools for coping and stress management comprised the majority of key app events (n=325,691, 70.4%), and the majority of app users tried a tool for managing stress (n=28,009, 58.8%). COVID Coach was utilized for ≤3 days by 80.9% (n=34,611) of the sample whose first day of app use occurred within the 6-month observation window. Usage of the key content in COVID Coach predicted returning to the app for a second day. Among those who tried at least one coping tool on their first day of app use, 57.2% (n=11,444) returned for a second visit; whereas only 46.3% (n=10,546) of those who did not try a tool returned (P<.001). Symptoms of anxiety, depression, and posttraumatic stress disorder (PTSD) were prevalent among app users. For example, among app users who completed an anxiety assessment on their first day of app use (n=4870, 11.4% of users), 55.1% (n=2680) reported levels of anxiety that were moderate to severe, and 29.9% (n=1455) of scores fell into the severe symptom range. On average, those with moderate levels of depression on their first day of app use returned to the app for a greater number of days (mean 3.72 days) than those with minimal symptoms (mean 3.08 days; t1=3.01, P=.003). Individuals with significant PTSD symptoms on their first day of app use utilized the app for a significantly greater number of days (mean 3.79 days) than those with fewer symptoms (mean 3.13 days; t1=2.29, P=.02). Conclusions: As the mental health impacts of the pandemic continue to be widespread and increasing, digital health resources, such as apps like COVID Coach, are a scalable way to provide evidence-informed tools and resources. Future research is needed to better understand for whom and under what conditions the app is most helpful and how to increase and sustain engagement.

  • Source: freepik; Copyright: tirachardz; URL:; License: Licensed by JMIR.

    The Reliability of Remote Patient-Reported Outcome Measures via Mobile Apps to Replace Outpatient Visits After Rotator Cuff Repair Surgery: Repetitive...


    Background: With the development of health care–related mobile apps, attempts have been made to implement remote patient-reported outcome measures (PROMs). In order for remote PROMs to be widely used by mobile apps, the results should not be different depending on the location; that is, remote PROM results performed in locations other than hospitals should be able to obtain reliable results equivalent to those performed in hospitals, and this is very important. However, to our knowledge, there are no studies that have assessed the reliability of PROMs using mobile apps according to the location by comparing the results performed remotely from the hospital and performed at the outpatient visits. Objective: The purpose of this study was to evaluate the reliability of remote PROMs using mobile apps compared to PROMs performed during outpatient follow-up visits after arthroscopic shoulder surgery. Methods: A total of 174 patients who underwent arthroscopic rotator cuff repair completed questionnaires 2 days before visiting the clinic for the 1-, 2-, 3-, 6-, and 12-month follow-ups (test A). The patients completed the questionnaires at the clinic (test B) using the same mobile app and device for the 1-, 2-, 3-, 6-, and 12-month follow-ups. Test-retest comparisons were performed to analyze the differences and reliability of the PROMs according to the period. Results: Comparisons of tests A and B showed statistically significant differences at 1, 2, and 3 months (all Ps<.05 except for the ASES function scale at 3-months) but not 6 or 12 months after surgery (all Ps>.05). The intraclass correlation values between the two groups were relatively low at the 1-, 2-, and 3-month follow-ups but were within the reliable range at 6 and 12 months after surgery. The rate of completion of tests A and B using the mobile app was significantly lower in the group older than 70 years than in the other groups for all postoperative periods (P<.001). Conclusions: PROMs using mobile apps with different locations differed soon after surgery but were reliably similar after 6 months. The remote PROMs using mobile apps could be used reliably for the patient more than 6 months after surgery. However, it is to be expected that the use of mobile app–based questionnaires is not as useful in the group older than 70 years as in other age groups.

  • Source: FlickR; Copyright: Peter Wong; URL:; License: Creative Commons Attribution + Noncommercial + ShareAlike (CC-BY-NC-SA).

    Influence of Health Beliefs on Adherence to COVID-19 Preventative Practices: International, Social Media–Based Survey Study


    Background: Health behavior is influenced by culture and social context. However, there are limited data evaluating the scope of these influences on COVID-19 response. Objective: This study aimed to compare handwashing and social distancing practices in different countries and evaluate practice predictors using the health belief model (HBM). Methods: From April 11 to May 1, 2020, we conducted an online, cross-sectional survey disseminated internationally via social media. Participants were adults aged 18 years or older from four different countries: the United States, Mexico, Hong Kong (China), and Taiwan. Primary outcomes were self-reported handwashing and social distancing practices during COVID-19. Predictors included constructs of the HBM: perceived susceptibility, perceived severity, perceived benefits, perceived barriers, self-efficacy, and cues to action. Associations of these constructs with behavioral outcomes were assessed by multivariable logistic regression. Results: We analyzed a total of 71,851 participants, with 3070 from the United States, 3946 from Mexico, 1201 from Hong Kong (China), and 63,634 from Taiwan. Of these countries, respondents from the United States adhered to the most social distancing practices (χ23=2169.7, P<.001), while respondents from Taiwan performed the most handwashing (χ23=309.8, P<.001). Multivariable logistic regression analyses indicated that self-efficacy was a positive predictor for handwashing (odds ratio [OR]United States 1.58, 95% CI 1.21-2.07; ORMexico 1.5, 95% CI 1.21-1.96; ORHong Kong 2.48, 95% CI 1.80-3.44; ORTaiwan 2.30, 95% CI 2.21-2.39) and social distancing practices (ORUnited States 1.77, 95% CI 1.24-2.49; ORMexico 1.77, 95% CI 1.40-2.25; ORHong Kong 3.25, 95% CI 2.32-4.62; ORTaiwan 2.58, 95% CI 2.47-2.68) in all countries. Handwashing was positively associated with perceived susceptibility in Mexico, Hong Kong, and Taiwan, while social distancing was positively associated with perceived severity in the United States, Mexico, and Taiwan. Conclusions: Social media recruitment strategies can be used to reach a large audience during a pandemic. Self-efficacy was the strongest predictor for handwashing and social distancing. Policies that address relevant health beliefs can facilitate adoption of necessary actions for preventing COVID-19. Our findings may be explained by the timing of government policies, the number of cases reported in each country, individual beliefs, and cultural context.

  • Source: The Authors / Max Pixel; Copyright: The Authors / Max Pixel; URL:; License: Creative Commons Attribution (CC-BY).

    Using Automated Machine Learning to Predict the Mortality of Patients With COVID-19: Prediction Model Development Study


    Background: During a pandemic, it is important for clinicians to stratify patients and decide who receives limited medical resources. Machine learning models have been proposed to accurately predict COVID-19 disease severity. Previous studies have typically tested only one machine learning algorithm and limited performance evaluation to area under the curve analysis. To obtain the best results possible, it may be important to test different machine learning algorithms to find the best prediction model. Objective: In this study, we aimed to use automated machine learning (autoML) to train various machine learning algorithms. We selected the model that best predicted patients’ chances of surviving a SARS-CoV-2 infection. In addition, we identified which variables (ie, vital signs, biomarkers, comorbidities, etc) were the most influential in generating an accurate model. Methods: Data were retrospectively collected from all patients who tested positive for COVID-19 at our institution between March 1 and July 3, 2020. We collected 48 variables from each patient within 36 hours before or after the index time (ie, real-time polymerase chain reaction positivity). Patients were followed for 30 days or until death. Patients’ data were used to build 20 machine learning models with various algorithms via autoML. The performance of machine learning models was measured by analyzing the area under the precision-recall curve (AUPCR). Subsequently, we established model interpretability via Shapley additive explanation and partial dependence plots to identify and rank variables that drove model predictions. Afterward, we conducted dimensionality reduction to extract the 10 most influential variables. AutoML models were retrained by only using these 10 variables, and the output models were evaluated against the model that used 48 variables. Results: Data from 4313 patients were used to develop the models. The best model that was generated by using autoML and 48 variables was the stacked ensemble model (AUPRC=0.807). The two best independent models were the gradient boost machine and extreme gradient boost models, which had an AUPRC of 0.803 and 0.793, respectively. The deep learning model (AUPRC=0.73) was substantially inferior to the other models. The 10 most influential variables for generating high-performing models were systolic and diastolic blood pressure, age, pulse oximetry level, blood urea nitrogen level, lactate dehydrogenase level, D-dimer level, troponin level, respiratory rate, and Charlson comorbidity score. After the autoML models were retrained with these 10 variables, the stacked ensemble model still had the best performance (AUPRC=0.791). Conclusions: We used autoML to develop high-performing models that predicted the survival of patients with COVID-19. In addition, we identified important variables that correlated with mortality. This is proof of concept that autoML is an efficient, effective, and informative method for generating machine learning–based clinical decision support tools.

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Latest Submissions Open for Peer-Review:

View All Open Peer Review Articles
  • Adverse drug reaction analysis using statistical and machine learning methods: A systematic review

    Date Submitted: Feb 25, 2021

    Open Peer Review Period: Feb 25, 2021 - Apr 22, 2021

    An adverse drug reaction (ADR) is an unintended response induced by a drug. It is important to determine the association between drugs and ADRs. There are many methods to demonstrate this association....

    An adverse drug reaction (ADR) is an unintended response induced by a drug. It is important to determine the association between drugs and ADRs. There are many methods to demonstrate this association. This systematic review aimed to examine the analysis tools by considering original articles that introduced statistical and machine learning methods for predicting ADRs in humans. A systematic literature review of EMBASE and PubMed was conducted based on articles published from January 2015 to March 2020. The keywords were statistical, machine learning, and deep learning methods for the detection of ADR signals in the title and abstract. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis statement guidelines. In total, 72 articles were included in the current systematic review; of these, 51 and 21 addressed statistical and machine learning methods, respectively. This study provides a graphical overview of data-driven methods for detecting ADRs with multiple data sources for patient drug safety.

  • A Smartphone Application can Yield Accurate and Rapid Diagnosis of Acute Anterior Intracranial Arterial Occlusion

    Date Submitted: Feb 24, 2021

    Open Peer Review Period: Feb 24, 2021 - Apr 21, 2021

    Background: Telestroke have rapidly developed as a way to reach out to patients who are eligible for the administration of reperfusion therapy. Objective: The aim is to investigate if vascular neurolo...

    Background: Telestroke have rapidly developed as a way to reach out to patients who are eligible for the administration of reperfusion therapy. Objective: The aim is to investigate if vascular neurologists can make a quick and precise diagnosis of an intracranial large vessel occlusion (LVO) using a smartphone as well as a hospital desktop personal computer (PC) monitor. Methods: MRIs of acute ischemic stroke consecutive 108 patients with a territory of middle cerebral artery within 24 hours of onset were retrospectively enrolled. A LVO was estimated at the internal carotid artery and the middle cerebral artery (M1, M2 and M3). After blinding all clinical information, two vascular neurologists evaluated the presence or absence of LVO on MRA and FLAIR by using a smartphone (Smartphone-LVO decision) and a hospital desktop PC monitor (PC-LVO decision), independently. In order to analyze inter-device variability comparisons (Smartphone-LVO decision and PC-LVO decision) κ statistics were conducted. Image interpretation times between a Smartphone-LVO decision and a PC-LVO decision were compared. Results: Regarding the presence or absence of arterial occlusion, Smartphone-LVO decision broadly agreed with PC-LVO decision (vascular neurologist 1, κ=0.94, p<0.001; vascular neurologist 2, κ=0.89, p<0.001). Interpretation time of the Smartphone-LVO decision was similar to that of the PC-LVO decision. Conclusions: The smartphone application can yield an accurate diagnosis of anterior intracranial arterial occlusion patients in and outside the hospital. It should play an important role in organizing the stroke team for hyper-acute ischemic stroke.

  • Teleconsultation in the management of elective orthopaedic and spinal conditions during the COVID-19 pandemic: A prospective cohort study of patient experiences

    Date Submitted: Feb 22, 2021

    Open Peer Review Period: Feb 22, 2021 - Apr 19, 2021

    Background: The global adoption of teleconsultation has been expedited as a result of the COVID-19 pandemic. By allowing remote communication, teleconsultation may help limit the spread of the virus w...

    Background: The global adoption of teleconsultation has been expedited as a result of the COVID-19 pandemic. By allowing remote communication, teleconsultation may help limit the spread of the virus while maintaining the crucial patient-provider relationship. Objective: To evaluate the value of teleconsultation compared to in-person visits in the management of elective orthopaedic and spinal procedures. Methods: This was a prospective observational cohort study of 853 patients receiving orthopaedic and spinal care at a private outpatient clinic in New Zealand. Patients were randomly divided into two groups: (1) patients receiving telephone consultation remotely; and (2) patients receiving in-person office consultations at the outpatient clinic. All patients received telephone consultations for four weeks during the mandated COVID-19 lockdown, followed by four weeks of telephone or in-person consultation. Patient preference, satisfaction, and duration of visit were recorded. Comparisons of patient preference between groups, visit type, sex, and location were performed using Chi-square tests; similarly, satisfaction scores and visit durations were compared using a general linear model. Results: We report that 91% of patients in the telephone group preferred teleconsultation over in-person office visits during the COVID-19 lockdown (p=0.000). A combined-group analysis shows that 55.3% of all patients preferred teleconsultation compared to 31.2% who preferred in-person office visits (p=0.000). Patients in the telephone group reported significantly higher satisfaction scores (9.95 +/- 0.04, 95% CI [9.87-10.03]) compared to patients in the in-person group (9.53 +/- 0.04, 95% CI [9.45-9.62]; p=0.000). Additionally, in-person consultations were significantly longer in duration compared to telephone consultations, with a mean visit time of 6.70 min +/-0.18, 95% CI [6.32-7.02] compared to 5.10 min +/-0.17, 95% CI [4.73-5.42], respectively (p=0.000). Conclusions: Patients who utilize telephone consultations are more likely to prefer it over traditional, in-person visits in the future. This increased preference, coupled with higher patient satisfaction scores and shorter duration of visits, suggests that teleconsultation has a role in orthopaedic surgery, which may even extend beyond the COVID-19 pandemic. Clinical Trial: N/A

  • Anticipated benefits and concerns about sharing hospital outpatient visit notes with patients in the Netherlands: a mixed-methods study

    Date Submitted: Feb 5, 2021

    Open Peer Review Period: Feb 5, 2021 - Apr 2, 2021

    Background: The past few years have seen a rise in interest in sharing visit notes with patients, or "Open Notes". Objective: We sought to gather opinions about sharing outpatient clinic visit notes f...

    Background: The past few years have seen a rise in interest in sharing visit notes with patients, or "Open Notes". Objective: We sought to gather opinions about sharing outpatient clinic visit notes from patients and hospital physicians in the Netherlands, where there is currently no policy or incentive plan for shared visit notes. Methods: We conducted a survey of patients and doctors, as well as "think aloud" interviews to elicit more insight into the reasons behind participants' answers. Results: We surveyed 350 physicians and 90 patients, and interviews were conducted with an additional 13 physicians and 6 patients. A majority of patients (77%) were interested in viewing their visit notes, while a majority of physicians (82%) were opposed. A majority of patients (60%) expected the notes to be written in layman's language, but most physicians (60%) did not want to change their writing style to make it more understandable for patients. Doctors raised concerns that reading the note would make patients feel confused and anxious, that the patient would not understand the note, and that shared notes would result in more documentation time or losing a way to communicate with colleagues. Interviews also revealed concerns about documenting sensitive topics such as suspected abuse, and unlikely but worrisome differential diagnoses. Physicians also raised concerns about fragmenting the patient record. Patients also were uncertain if they would understand the notes (51%), and in interviews raised questions about security and privacy. Physicians did anticipate some benefits, such as better patient recall of the visit, shared decision-making, and keeping patients informed, but 24% indicated that they saw no benefit. Patients anticipated that they would remember the visit better, feel more in control, and better understand their health. Conclusions: Dutch patients are interested in shared visit notes, but physicians have many concerns which should be addressed if shared notes are pursued. In hospitals where shared notes are implemented, the effects should be monitored (objectively, if possible) to determine if the concerns raised by our participants have actualized into problems and whether the anticipated benefits are being realized. Clinical Trial: n/a

  • Machine Learning Algorithms to Detect Subclinical Keratoconus: Systematic Review

    Date Submitted: Jan 25, 2021

    Open Peer Review Period: Jan 24, 2021 - Mar 21, 2021

    Background: Keratoconus is a disorder characterized by progressive thinning and distortion of the cornea. If detected at an early stage corneal collagen cross linking can prevent disease progression a...

    Background: Keratoconus is a disorder characterized by progressive thinning and distortion of the cornea. If detected at an early stage corneal collagen cross linking can prevent disease progression and further visual loss. Whilst advanced forms are easily detected, reliably identifying subclinical disease can be problematic. A number of different machine learning algorithms have been used to improve the detection of subclinical keratoconus based on the analysis of single or multiple clinical measures such as corneal imaging, aberrometry, or biomechanical measurements. Objective: To survey and critically evaluate the literature on algorithmic detection of subclinical keratoconus and equivalent definitions. Methods: We performed a structured search of the following databases: Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica Database (EMBASE), Web of Science and Cochrane from Jan 1, 2010 to Oct 31, 2020. We included all full text studies that have used algorithms for the detection of subclinical keratoconus. We excluded studies that did not perform validation. Results: We compared the parameters measured and the design of the machine learning algorithms reported in 26 papers that met the inclusion criteria. All salient information required for detailed comparison including diagnostic criteria, demographic data, sample size, acquisition system, validation details, parameter inputs, machine learning algorithm and key results are reported in this study. Conclusions: Machine learning has the potential to improve the detection of subclinical keratoconus or early keratoconus in routine ophthalmic practice. Presently there is no consensus regarding the corneal parameters that should be included for assessment and the optimal design for the machine learning algorithm. We have identified avenues for further research to improve early detection and stratification of patients for early intervention to prevent disease progression. Clinical Trial: N/A