Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Journal Description

The Journal of Medical Internet Research (JMIR), now in its 21st year, is the pioneer open access eHealth journal and is the flagship journal of JMIR Publications. It is the leading digital health journal globally in terms of quality/visibility (Impact Factor 2019: 5.03), ranking Q1 in the medical informatics category, and is also the largest journal in the field. The journal focuses on emerging technologies, medical devices, apps, engineering, telehealth and informatics applications for patient education, prevention, population health and clinical care. As a leading high-impact journal in its disciplines (health informatics and health services research), it is selective, but it is now complemented by almost 30 specialty JMIR sister journals, which have a broader scope, and which together receive over 6.000 submissions a year. Peer-review reports are portable across JMIR journals and papers can be transferred, so authors save time by not having to resubmit a paper to different journal but can simply transfer it between journals. 

As an open access journal, we are read by clinicians, allied health professionals, informal caregivers, and patients alike, and have (as with all JMIR journals) a focus on readable and applied science reporting the design and evaluation of health innovations and emerging technologies. We publish original research, viewpoints, and reviews (both literature reviews and medical device/technology/app reviews).

We are also a leader in participatory and open science approaches, and offer the option to publish new submissions immediately as preprints, which receive DOIs for immediate citation (eg, in grant proposals), and for open peer-review purposes. We also invite patients to participate (eg, as peer-reviewers) and have patient representatives on editorial boards.

Be a widely cited leader in the digitial health revolution and submit your paper today!


Recent Articles:

  • Source: Rawpixel; Copyright:; URL:; License: Licensed by JMIR.

    A Comprehensive Overview of the COVID-19 Literature: Machine Learning–Based Bibliometric Analysis


    Background: Shortly after the emergence of COVID-19, researchers rapidly mobilized to study numerous aspects of the disease such as its evolution, clinical manifestations, effects, treatments, and vaccinations. This led to a rapid increase in the number of COVID-19–related publications. Identifying trends and areas of interest using traditional review methods (eg, scoping and systematic reviews) for such a large domain area is challenging. Objective: We aimed to conduct an extensive bibliometric analysis to provide a comprehensive overview of the COVID-19 literature. Methods: We used the COVID-19 Open Research Dataset (CORD-19) that consists of a large number of research articles related to all coronaviruses. We used a machine learning–based method to analyze the most relevant COVID-19–related articles and extracted the most prominent topics. Specifically, we used a clustering algorithm to group published articles based on the similarity of their abstracts to identify research hotspots and current research directions. We have made our software accessible to the community via GitHub. Results: Of the 196,630 publications retrieved from the database, we included 28,904 in our analysis. The mean number of weekly publications was 990 (SD 789.3). The country that published the highest number of COVID-19–related articles was China (2950/17,270, 17.08%). The highest number of articles were published in bioRxiv. Lei Liu affiliated with the Southern University of Science and Technology in China published the highest number of articles (n=46). Based on titles and abstracts alone, we were able to identify 1515 surveys, 733 systematic reviews, 512 cohort studies, 480 meta-analyses, and 362 randomized control trials. We identified 19 different topics covered among the publications reviewed. The most dominant topic was public health response, followed by clinical care practices during the COVID-19 pandemic, clinical characteristics and risk factors, and epidemic models for its spread. Conclusions: We provide an overview of the COVID-19 literature and have identified current hotspots and research directions. Our findings can be useful for the research community to help prioritize research needs and recognize leading COVID-19 researchers, institutes, countries, and publishers. Our study shows that an AI-based bibliometric analysis has the potential to rapidly explore a large corpus of academic publications during a public health crisis. We believe that this work can be used to analyze other eHealth-related literature to help clinicians, administrators, and policy makers to obtain a holistic view of the literature and be able to categorize different topics of the existing research for further analyses. It can be further scaled (for instance, in time) to clinical summary documentation. Publishers should avoid noise in the data by developing a way to trace the evolution of individual publications and unique authors.

  • Source: Adobe Stock; Copyright: Derek Yung; URL:; License: Licensed by JMIR.

    Comparing Public Perceptions and Preventive Behaviors During the Early Phase of the COVID-19 Pandemic in Hong Kong and the United Kingdom: Cross-sectional...


    Background: Given the public health responses to previous respiratory disease pandemics, and in the absence of treatments and vaccines, the mitigation of the COVID-19 pandemic relies on population engagement in nonpharmaceutical interventions. This engagement is largely driven by risk perception, anxiety levels, and knowledge, as well as by historical exposure to disease outbreaks, government responses, and cultural factors. Objective: The aim of this study is to compare psychobehavioral responses in Hong Kong and the United Kingdom during the early phase of the COVID-19 pandemic. Methods: Comparable cross-sectional surveys were administered to adults in Hong Kong and the United Kingdom during the early phase of the epidemic in each setting. Explanatory variables included demographics, risk perception, knowledge of COVID-19, anxiety level, and preventive behaviors. Responses were weighted according to census data. Logistic regression models, including effect modification to quantify setting differences, were used to assess the association between the explanatory variables and the adoption of social distancing measures. Results: Data from 3431 complete responses (Hong Kong, 1663; United Kingdom, 1768) were analyzed. Perceived severity of symptoms differed by setting, with weighted percentages of 96.8% for Hong Kong (1621/1663) and 19.9% for the United Kingdom (366/1768). A large proportion of respondents were abnormally or borderline anxious (Hong Kong: 1077/1603, 60.0%; United Kingdom: 812/1768, 46.5%) and regarded direct contact with infected individuals as the transmission route of COVID-19 (Hong Kong: 94.0%-98.5%; United Kingdom: 69.2%-93.5%; all percentages weighted), with Hong Kong identifying additional routes. Hong Kong reported high levels of adoption of various social distancing measures (Hong Kong: 32.6%-93.7%; United Kingdom: 17.6%-59.0%) and mask-wearing (Hong Kong: 98.8% (1647/1663); United Kingdom: 3.1% (53/1768)). The impact of perceived severity of symptoms and perceived ease of transmission of COVID-19 on the adoption of social distancing measures varied by setting. In Hong Kong, these factors had no impact, whereas in the United Kingdom, those who perceived their symptom severity as “high” were more likely to adopt social distancing (adjusted odds ratios [aORs] 1.58-3.01), and those who perceived transmission as “easy” were prone to adopt both general social distancing (aOR 2.00, 95% CI 1.57-2.55) and contact avoidance (aOR 1.80, 95% CI 1.41-2.30). The impact of anxiety on adopting social distancing did not vary by setting. Conclusions: Our results suggest that health officials should ascertain baseline levels of risk perception and knowledge in populations, as well as prior sensitization to infectious disease outbreaks, during the development of mitigation strategies. Risk should be communicated through suitable media channels—and trust should be maintained—while early intervention remains the cornerstone of effective outbreak response.

  • Source: Pixabay; Copyright: Free pixabay license; URL:; License: Public Domain (CC0).

    Designing the Optimal Digital Health Intervention for Patients’ Use Before and After Elective Orthopedic Surgery: Qualitative Study


    Background: Health behavior changes made by patients during the perioperative period can impact the outcomes and success of elective surgeries. However, there remains a limited understanding of how best to support patients during this time, particularly through the use of digital health interventions. Recognizing and understanding the potential unmet needs of elective orthopedic surgery patients is central to motivating healthier behavior change, improving recovery, and optimizing overall surgical success in the short and long term. Objective: The aim of this study is to explore patient perspectives on technology features that would help support them to change their lifestyle behaviors during the pre- and postoperative periods, and that could potentially maintain long-term healthy lifestyles following recovery. Methods: Semistructured interviews with pre- and postoperative elective orthopedic patients were conducted between May and June 2020 using telephone and video call–based software. Patient perspectives on the use of digital technologies to complement current surgical care and support with lifestyle behavior changes were discussed. Interviews were audio recorded and transcribed verbatim. Reflexive thematic analysis enabled the development of themes from the data, with QSR NVivo software (version 12) facilitating data management. Ethical approval was obtained from the National Health Service Health Research Authority. Results: A total of 18 participants were interviewed. Four themes were developed from the data regarding the design and functionality of digital technologies to best support the perioperative journey. These center around an intervention’s ability to incorporate interactive, user-centered features; direct a descriptive and structured recovery; enable customizable, patient-controlled settings; and deliver both general and specific surgical advice in a timely manner. Interventions that are initiated preoperatively and continued postoperatively were perceived as beneficial. Interventions designed with personalized milestones were found to better guide patients through a structured recovery. Individualized tailoring of preparatory and recovery information was desired by patients with previously high levels of physical activity before surgery. The use of personalized progression-based exercises further encouraged physical recovery; game-like rewards and incentives were regarded as motivational for making and sustaining health behavior change. In-built video calling and messaging features offered connectivity with peers and clinicians for supported care delivery. Conclusions: Specific intervention design and functionality features can provide better, structured support for elective orthopedic patients across the entire surgical journey and beyond. This study provides much-needed evidence relating to the optimal design and timing of digital interventions for elective orthopedic surgical patients. Findings from this study suggest a desire for personalized perioperative care, in turn, supporting patients to make health behavior changes to optimize surgical success. These findings should be used to influence future co-design projects to enable the design and implementation of patient-focused, tailored, and targeted digital health technologies within modern health care settings.

  • Source: Pexels; Copyright: mikoto.raw; URL:; License: Licensed by JMIR.

    Machine Learning for Mental Health in Social Media: Bibliometric Study


    Background: Social media platforms provide an easily accessible and time-saving communication approach for individuals with mental disorders compared to face-to-face meetings with medical providers. Recently, machine learning (ML)-based mental health exploration using large-scale social media data has attracted significant attention. Objective: We aimed to provide a bibliometric analysis and discussion on research trends of ML for mental health in social media. Methods: Publications addressing social media and ML in the field of mental health were retrieved from the Scopus and Web of Science databases. We analyzed the publication distribution to measure productivity on sources, countries, institutions, authors, and research subjects, and visualized the trends in this field using a keyword co-occurrence network. The research methodologies of previous studies with high citations are also thoroughly described. Results: We obtained a total of 565 relevant papers published from 2015 to 2020. In the last 5 years, the number of publications has demonstrated continuous growth with Lecture Notes in Computer Science and Journal of Medical Internet Research as the two most productive sources based on Scopus and Web of Science records. In addition, notable methodological approaches with data resources presented in high-ranking publications were investigated. Conclusions: The results of this study highlight continuous growth in this research area. Moreover, we retrieved three main discussion points from a comprehensive overview of highly cited publications that provide new in-depth directions for both researchers and practitioners.

  • Source: Freepik; Copyright:; URL:; License: Licensed by JMIR.

    Natural Language Processing and Machine Learning for Identifying Incident Stroke From Electronic Health Records: Algorithm Development and Validation


    Background: Stroke is an important clinical outcome in cardiovascular research. However, the ascertainment of incident stroke is typically accomplished via time-consuming manual chart abstraction. Current phenotyping efforts using electronic health records for stroke focus on case ascertainment rather than incident disease, which requires knowledge of the temporal sequence of events. Objective: The aim of this study was to develop a machine learning–based phenotyping algorithm for incident stroke ascertainment based on diagnosis codes, procedure codes, and clinical concepts extracted from clinical notes using natural language processing. Methods: The algorithm was trained and validated using an existing epidemiology cohort consisting of 4914 patients with atrial fibrillation (AF) with manually curated incident stroke events. Various combinations of feature sets and machine learning classifiers were compared. Using a heuristic rule based on the composition of concepts and codes, we further detected the stroke subtype (ischemic stroke/transient ischemic attack or hemorrhagic stroke) of each identified stroke. The algorithm was further validated using a cohort (n=150) stratified sampled from a population in Olmsted County, Minnesota (N=74,314). Results: Among the 4914 patients with AF, 740 had validated incident stroke events. The best-performing stroke phenotyping algorithm used clinical concepts, diagnosis codes, and procedure codes as features in a random forest classifier. Among patients with stroke codes in the general population sample, the best-performing model achieved a positive predictive value of 86% (43/50; 95% CI 0.74-0.93) and a negative predictive value of 96% (96/100). For subtype identification, we achieved an accuracy of 83% in the AF cohort and 80% in the general population sample. Conclusions: We developed and validated a machine learning–based algorithm that performed well for identifying incident stroke and for determining type of stroke. The algorithm also performed well on a sample from a general population, further demonstrating its generalizability and potential for adoption by other institutions.

  • Source: Freepik; Copyright: Freepik; URL:; License: Licensed by JMIR.

    Understanding Concerns, Sentiments, and Disparities Among Population Groups During the COVID-19 Pandemic Via Twitter Data Mining: Large-scale Cross-sectional...


    Background: Since the beginning of the COVID-19 pandemic in late 2019, its far-reaching impacts have been witnessed globally across all aspects of human life, such as health, economy, politics, and education. Such widely penetrating impacts cast significant and profound burdens on all population groups, incurring varied concerns and sentiments among them. Objective: This study aims to identify the concerns, sentiments, and disparities of various population groups during the COVID-19 pandemic through a cross-sectional study conducted via large-scale Twitter data mining infoveillance. Methods: This study consisted of three steps: first, tweets posted during the pandemic were collected and preprocessed on a large scale; second, the key population attributes, concerns, sentiments, and emotions were extracted via a collection of natural language processing procedures; third, multiple analyses were conducted to reveal concerns, sentiments, and disparities among population groups during the pandemic. Overall, this study implemented a quick, effective, and economical approach for analyzing population-level disparities during a public health event. The source code developed in this study was released for free public use at GitHub. Results: A total of 1,015,655 original English tweets posted from August 7 to 12, 2020, were acquired and analyzed to obtain the following results. Organizations were significantly more concerned about COVID-19 (odds ratio [OR] 3.48, 95% CI 3.39-3.58) and expressed more fear and depression emotions than individuals. Females were less concerned about COVID-19 (OR 0.73, 95% CI 0.71-0.75) and expressed less fear and depression emotions than males. Among all age groups (ie, ≤18, 19-29, 30-39, and ≥40 years of age), the attention ORs of COVID-19 fear and depression increased significantly with age. It is worth noting that not all females paid less attention to COVID-19 than males. In the age group of 40 years or older, females were more concerned than males, especially regarding the economic and education topics. In addition, males 40 years or older and 18 years or younger were the least positive. Lastly, in all sentiment analyses, the sentiment polarities regarding political topics were always the lowest among the five topics of concern across all population groups. Conclusions: Through large-scale Twitter data mining, this study revealed that meaningful differences regarding concerns and sentiments about COVID-19-related topics existed among population groups during the study period. Therefore, specialized and varied attention and support are needed for different population groups. In addition, the efficient analysis method implemented by our publicly released code can be utilized to dynamically track the evolution of each population group during the pandemic or any other major event for better informed public health research and interventions.

  • Source: Adobe Stock; Copyright: zapp2photo; URL:; License: Licensed by JMIR.

    Future Medical Artificial Intelligence Application Requirements and Expectations of Physicians in German University Hospitals: Web-Based Survey


    Background: The increasing development of artificial intelligence (AI) systems in medicine driven by researchers and entrepreneurs goes along with enormous expectations for medical care advancement. AI might change the clinical practice of physicians from almost all medical disciplines and in most areas of health care. While expectations for AI in medicine are high, practical implementations of AI for clinical practice are still scarce in Germany. Moreover, physicians’ requirements and expectations of AI in medicine and their opinion on the usage of anonymized patient data for clinical and biomedical research have not been investigated widely in German university hospitals. Objective: This study aimed to evaluate physicians’ requirements and expectations of AI in medicine and their opinion on the secondary usage of patient data for (bio)medical research (eg, for the development of machine learning algorithms) in university hospitals in Germany. Methods: A web-based survey was conducted addressing physicians of all medical disciplines in 8 German university hospitals. Answers were given using Likert scales and general demographic responses. Physicians were asked to participate locally via email in the respective hospitals. Results: The online survey was completed by 303 physicians (female: 121/303, 39.9%; male: 173/303, 57.1%; no response: 9/303, 3.0%) from a wide range of medical disciplines and work experience levels. Most respondents either had a positive (130/303, 42.9%) or a very positive attitude (82/303, 27.1%) towards AI in medicine. There was a significant association between the personal rating of AI in medicine and the self-reported technical affinity level (H4=48.3, P<.001). A vast majority of physicians expected the future of medicine to be a mix of human and artificial intelligence (273/303, 90.1%) but also requested a scientific evaluation before the routine implementation of AI-based systems (276/303, 91.1%). Physicians were most optimistic that AI applications would identify drug interactions (280/303, 92.4%) to improve patient care substantially but were quite reserved regarding AI-supported diagnosis of psychiatric diseases (62/303, 20.5%). Of the respondents, 82.5% (250/303) agreed that there should be open access to anonymized patient databases for medical and biomedical research. Conclusions: Physicians in stationary patient care in German university hospitals show a generally positive attitude towards using most AI applications in medicine. Along with this optimism comes several expectations and hopes that AI will assist physicians in clinical decision making. Especially in fields of medicine where huge amounts of data are processed (eg, imaging procedures in radiology and pathology) or data are collected continuously (eg, cardiology and intensive care medicine), physicians’ expectations of AI to substantially improve future patient care are high. In the study, the greatest potential was seen in the application of AI for the identification of drug interactions, assumedly due to the rising complexity of drug administration to polymorbid, polypharmacy patients. However, for the practical usage of AI in health care, regulatory and organizational challenges still have to be mastered.

  • Source: iStock; Copyright: Tero Vesalainen; URL:; License: Licensed by the authors.

    Linguistic Analysis of Online Communication About a Novel Persecutory Belief System (Gangstalking): Mixed Methods Study


    Background: Gangstalking is a novel persecutory belief system whereby those affected believe they are being followed, stalked, and harassed by a large number of people, often numbering in the thousands. The harassment is experienced as an accretion of innumerable individually benign acts such as people clearing their throat, muttering under their breath, or giving dirty looks as they pass on the street. Individuals affected by this belief system congregate in online fora to seek support, share experiences, and interact with other like-minded individuals. Such people identify themselves as targeted individuals. Objective: The objective of the study was to characterize the linguistic and rhetorical practices used by contributors to the gangstalking forum to construct, develop, and contest the gangstalking belief system. Methods: This mixed methods study employed corpus linguistics, which involves using computational techniques to examine recurring linguistic patterns in large, digitized bodies of authentic language data. Discourse analysis is an approach to text analysis which focuses on the ways in which linguistic choices made by text creators contribute to particular functions and representations. We assembled a 225,000-word corpus of postings on a gangstalking support forum. We analyzed these data using keyword analysis, collocation analysis, and manual examination of concordances to identify discursive and rhetorical practices among self-identified targeted individuals. Results: The gangstalking forum served as a site of discursive contest between 2 opposing worldviews. One is that gangstalking is a widespread, insidious, and centrally coordinated system of persecution employing community members, figures of authority, and state actors. This was the dominant discourse in the study corpus. The opposing view is a medicalized discourse supporting gangstalking as a form of mental disorder. Contributors used linguistic practices such as presupposition, nominalization, and the use of specialized jargon to construct gangstalking as real and external to the individual affected. Although contributors generally rejected the notion that they were affected by mental disorder, in some instances, they did label others in the forum as impacted/affected by mental illness if their accounts if their accounts were deemed to be too extreme or bizarre. Those affected demonstrated a concern with accumulating evidence to prove their position to incredulous others. Conclusions: The study found that contributors to the study corpus accomplished a number of tasks. They used linguistic practices to co-construct an internally coherent and systematized persecutory belief system. They advanced a position that gangstalking is real and contested the medicalizing discourse that gangstalking is a form of mental disorder. They supported one another by sharing similar experiences and providing encouragement and advice. Finally, they commiserated over the challenges of proving the existence of gangstalking.

  • Source: Unsplash; Copyright: René Ranisch; URL:; License: Licensed by JMIR.

    Association of Spontaneous and Induced Self-Affirmation With Smoking Cessation in Users of a Mobile App: Randomized Controlled Trial


    Background: Most smokers attempt to stop using cigarettes numerous times before successfully quitting. Cigarette cravings may undermine perceived competence to quit and thus constitute psychological threats to the individual’s self-concept. Self-affirmation may promote smoking cessation by offsetting these threats. Objective: This study examines whether self-affirmation is associated with smoking cessation in the context of a cessation app. Two types of self-affirmation are examined: tendency to spontaneously self-affirm, and self-affirmation inductions added to a publicly available smoking cessation app (Smoke-Free Quit Smoking Now). In addition, this study explores whether optimism and emotional states (happiness, anger, anxiousness, hopefulness, sadness) predict smoking cessation. Methods: All users who met the inclusion criteria, provided consent to participate, and completed a baseline assessment, including all individual difference measures, were randomized to 1 of 4 conditions. Half of the participants were randomly assigned to complete a self-affirmation induction upon study entry. Orthogonally, half of the participants were randomly assigned to receive self-affirming text notifications during their quit attempt or to receive conventional notifications. The induction and the text notifications were fully automated, and all data were collected through self-assessments in the app. Self-reported smoking cessation was assessed 1 month and 3 months following study entry. Results: The study enrolled 7899 participants; 647 completed the 1-month follow-up. Using an intent-to-treat analysis at the 1-month follow-up, 7.2% (569/7899) of participants self-reported not smoking in the previous week and 6.4% (503/7899) self-reported not smoking in the previous month. Greater tendency to spontaneously self-affirm predicted a greater likelihood of cessation (P<.001) at 1 month after controlling for smoking-related variables. Neither self-affirmation induction influenced cessation. In addition, spontaneous self-affirmation did not moderate the relationship between self-affirmation inductions and cessation. Greater baseline sadness was associated with a lower likelihood of reporting successful cessation. Optimism predicted past-week cessation at the 1-month follow-up, and both happiness and anger predicted past-month cessation at the 1-month follow-up; however, none of these potential predictors moderated the relationship between self-affirmation conditions and successful cessation. Conclusions: Spontaneous self-affirmation may be an important psychological resource for managing threats to self-concept during the smoking cessation process. Sadness may hinder quit attempts. Future research can explicate how spontaneous versus induced self-affirmation can promote smoking cessation and examine boundary conditions for the effectiveness of disseminated self-affirmation interventions. Trial Registration: ISRCTN Registry 56646695;

  • Source: Image created by the authors; Copyright: The Authors; URL:; License: Creative Commons Attribution (CC-BY).

    Adoption of COVID-19 Contact Tracing Apps: A Balance Between Privacy and Effectiveness


    With the relative ubiquity of smartphones, contact tracing and exposure notification apps have been looked to as novel methods to help reduce the transmission of COVID-19. Many countries have created apps that lie across a spectrum from privacy-first approaches to those that have very few privacy measures. The level of privacy incorporated into an app is largely based on the societal norms and values of a particular country. Digital health technologies can be highly effective and preserve privacy at the same time, but in the case of contact tracing and exposure notification apps, there is a trade-off between increased privacy measures and the effectiveness of the app. In this article, examples from various countries are used to highlight how characteristics of contract tracing and exposure notification apps contribute to the perceived levels of privacy awarded to citizens and how this impacts an app’s effectiveness. We conclude that finding the right balance between privacy and effectiveness, while critical, is challenging because it is highly context-specific.

  • Source: Unsplash; Copyright: National Cancer Institute; URL:; License: Licensed by JMIR.

    Contribution of Free-Text Comments to the Burden of Documentation: Assessment and Analysis of Vital Sign Comments in Flowsheets


    Background: Documentation burden is a common problem with modern electronic health record (EHR) systems. To reduce this burden, various recording methods (eg, voice recorders or motion sensors) have been proposed. However, these solutions are in an early prototype phase and are unlikely to transition into practice in the near future. A more pragmatic alternative is to directly modify the implementation of the existing functionalities of an EHR system. Objective: This study aims to assess the nature of free-text comments entered into EHR flowsheets that supplement quantitative vital sign values and examine opportunities to simplify functionality and reduce documentation burden. Methods: We evaluated 209,055 vital sign comments in flowsheets that were generated in the Epic EHR system at the Vanderbilt University Medical Center in 2018. We applied topic modeling, as well as the natural language processing Clinical Language Annotation, Modeling, and Processing software system, to extract generally discussed topics and detailed medical terms (expressed as probability distribution) to investigate the stories communicated in these comments. Results: Our analysis showed that 63.33% (6053/9557) of the users who entered vital signs made at least one free-text comment in vital sign flowsheet entries. The user roles that were most likely to compose comments were registered nurse, technician, and licensed nurse. The most frequently identified topics were the notification of a result to health care providers (0.347), the context of a measurement (0.307), and an inability to obtain a vital sign (0.224). There were 4187 unique medical terms that were extracted from 46,029 (0.220) comments, including many symptom-related terms such as “pain,” “upset,” “dizziness,” “coughing,” “anxiety,” “distress,” and “fever” and drug-related terms such as “tylenol,” “anesthesia,” “cannula,” “oxygen,” “motrin,” “rituxan,” and “labetalol.” Conclusions: Considering that flowsheet comments are generally not displayed or automatically pulled into any clinical notes, our findings suggest that the flowsheet comment functionality can be simplified (eg, via structured response fields instead of a text input dialog) to reduce health care provider effort. Moreover, rich and clinically important medical terms such as medications and symptoms should be explicitly recorded in clinical notes for better visibility.

  • Source: Maximilian Fink; Copyright: The Authors; URL:; License: Creative Commons Attribution (CC-BY).

    Assessment of Diagnostic Competences With Standardized Patients Versus Virtual Patients: Experimental Study in the Context of History Taking


    Background: Standardized patients (SPs) have been one of the popular assessment methods in clinical teaching for decades, although they are resource intensive. Nowadays, simulated virtual patients (VPs) are increasingly used because they are permanently available and fully scalable to a large audience. However, empirical studies comparing the differential effects of these assessment methods are lacking. Similarly, the relationships between key variables associated with diagnostic competences (ie, diagnostic accuracy and evidence generation) in these assessment methods still require further research. Objective: The aim of this study is to compare perceived authenticity, cognitive load, and diagnostic competences in performance-based assessment using SPs and VPs. This study also aims to examine the relationships of perceived authenticity, cognitive load, and quality of evidence generation with diagnostic accuracy. Methods: We conducted an experimental study with 86 medical students (mean 26.03 years, SD 4.71) focusing on history taking in dyspnea cases. Participants solved three cases with SPs and three cases with VPs in this repeated measures study. After each case, students provided a diagnosis and rated perceived authenticity and cognitive load. The provided diagnosis was scored in terms of diagnostic accuracy; the questions asked by the medical students were rated with respect to their quality of evidence generation. In addition to regular null hypothesis testing, this study used equivalence testing to investigate the absence of meaningful effects. Results: Perceived authenticity (1-tailed t81=11.12; P<.001) was higher for SPs than for VPs. The correlation between diagnostic accuracy and perceived authenticity was very small (r=0.05) and neither equivalent (P=.09) nor statistically significant (P=.32). Cognitive load was equivalent in both assessment methods (t82=2.81; P=.003). Intrinsic cognitive load (1-tailed r=−0.30; P=.003) and extraneous load (1-tailed r=−0.29; P=.003) correlated negatively with the combined score for diagnostic accuracy. The quality of evidence generation was positively related to diagnostic accuracy for VPs (1-tailed r=0.38; P<.001); this finding did not hold for SPs (1-tailed r=0.05; P=.32). Comparing both assessment methods with each other, diagnostic accuracy was higher for SPs than for VPs (2-tailed t85=2.49; P=.01). Conclusions: The results on perceived authenticity demonstrate that learners experience SPs as more authentic than VPs. As higher amounts of intrinsic and extraneous cognitive loads are detrimental to performance, both types of cognitive load must be monitored and manipulated systematically in the assessment. Diagnostic accuracy was higher for SPs than for VPs, which could potentially negatively affect students’ grades with VPs. We identify and discuss possible reasons for this performance difference between both assessment methods.

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Latest Submissions Open for Peer-Review:

View All Open Peer Review Articles
  • What types of errors are hiding in Google Scholar data? Methodological concerns

    Date Submitted: Mar 2, 2021

    Open Peer Review Period: Mar 2, 2021 - Apr 27, 2021

    Background: Google Scholar (GS) is a free tool that may be used by researchers to analyze citations, to find appropriate literature or to evaluate the quality of an author or a contender for tenure, p...

    Background: Google Scholar (GS) is a free tool that may be used by researchers to analyze citations, to find appropriate literature or to evaluate the quality of an author or a contender for tenure, promotion, a faculty position, funding or research grants. GS has become a major bibliographic and citation database. Following the literature, databases such as PubMed, PsycINFO, Scopus or Web of Science can be used in place of GS because they are more reliable. Objective: The aim of this study is to examine the accuracy of citation data collected from GS and provide a comprehensive description of the errors and miscounts identified. Methods: 281 documents that cited two specific works were retrieved from the Publish or Perish software and examined. This work studied the false positive issue inherent in the analysis of neuroimaging data. Results: The results reveal an unprecedented error rate: 99.3% of the references examined contain at least one error. Consequently, Google Scholar data not only fail to be accurate but also potentially expose those researchers who would use these data without verification to substantial biases in their analyses and results. Conclusions: Google Scholar data not only fail to be accurate but also potentially expose those researchers who would use these data without verification to substantial biases in their analyses and results.