%0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 2 %P e16866 %T The Economic Impact of Artificial Intelligence in Health Care: Systematic Review %A Wolff,Justus %A Pauling,Josch %A Keck,Andreas %A Baumbach,Jan %+ TUM School of Life Sciences Weihenstephan, Technical University of Munich, Maximus-von-Imhof-Forum 3, Freising, 85354, Germany, 49 40329012 0, justus.wolff@syte-institute.com %K telemedicine %K artificial intelligence %K machine learning %K cost-benefit analysis %D 2020 %7 20.2.2020 %9 Review %J J Med Internet Res %G English %X Background: Positive economic impact is a key decision factor in making the case for or against investing in an artificial intelligence (AI) solution in the health care industry. It is most relevant for the care provider and insurer as well as for the pharmaceutical and medical technology sectors. Although the broad economic impact of digital health solutions in general has been assessed many times in literature and the benefit for patients and society has also been analyzed, the specific economic impact of AI in health care has been addressed only sporadically. Objective: This study aimed to systematically review and summarize the cost-effectiveness studies dedicated to AI in health care and to assess whether they meet the established quality criteria. Methods: In a first step, the quality criteria for economic impact studies were defined based on the established and adapted criteria schemes for cost impact assessments. In a second step, a systematic literature review based on qualitative and quantitative inclusion and exclusion criteria was conducted to identify relevant publications for an in-depth analysis of the economic impact assessment. In a final step, the quality of the identified economic impact studies was evaluated based on the defined quality criteria for cost-effectiveness studies. Results: Very few publications have thoroughly addressed the economic impact assessment, and the economic assessment quality of the reviewed publications on AI shows severe methodological deficits. Only 6 out of 66 publications could be included in the second step of the analysis based on the inclusion criteria. Out of these 6 studies, none comprised a methodologically complete cost impact analysis. There are two areas for improvement in future studies. First, the initial investment and operational costs for the AI infrastructure and service need to be included. Second, alternatives to achieve similar impact must be evaluated to provide a comprehensive comparison. Conclusions: This systematic literature analysis proved that the existing impact assessments show methodological deficits and that upcoming evaluations require more comprehensive economic analyses to enable economic decisions for or against implementing AI technology in health care. %R 10.2196/16866 %U http://www.jmir.org/2020/2/e16866/ %U https://doi.org/10.2196/16866 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 2 %P e17061 %T Detection of Postictal Generalized Electroencephalogram Suppression: Random Forest Approach %A Li,Xiaojin %A Tao,Shiqiang %A Jamal-Omidi,Shirin %A Huang,Yan %A Lhatoo,Samden D %A Zhang,Guo-Qiang %A Cui,Licong %+ School of Biomedical Informatics, University of Texas Health Science Center, 7000 Fannin St, Houston, TX, 77030, United States, 1 7135003791, licong.cui@uth.tmc.edu %K epilepsy %K generalized tonic-clonic seizure %K postictal generalized EEG suppression %K EEG %K random forest %D 2020 %7 14.2.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Sudden unexpected death in epilepsy (SUDEP) is second only to stroke in neurological events resulting in years of potential life lost. Postictal generalized electroencephalogram (EEG) suppression (PGES) is a period of suppressed brain activity often occurring after generalized tonic-clonic seizure, a most significant risk factor for SUDEP. Therefore, PGES has been considered as a potential biomarker for SUDEP risk. Automatic PGES detection tools can address the limitations of labor-intensive, and sometimes inconsistent, visual analysis. A successful approach to automatic PGES detection must overcome computational challenges involved in the detection of subtle amplitude changes in EEG recordings, which may contain physiological and acquisition artifacts. Objective: This study aimed to present a random forest approach for automatic PGES detection using multichannel human EEG recordings acquired in epilepsy monitoring units. Methods: We used a combination of temporal, frequency, wavelet, and interchannel correlation features derived from EEG signals to train a random forest classifier. We also constructed and applied confidence-based correction rules based on PGES state changes. Motivated by practical utility, we introduced a new, time distance–based evaluation method for assessing the performance of PGES detection algorithms. Results: The time distance–based evaluation showed that our approach achieved a 5-second tolerance-based positive prediction rate of 0.95 for artifact-free signals. For signals with different artifact levels, our prediction rates varied from 0.68 to 0.81. Conclusions: We introduced a feature-based, random forest approach for automatic PGES detection using multichannel EEG recordings. Our approach achieved increasingly better time distance–based performance with reduced signal artifact levels. Further study is needed for PGES detection algorithms to perform well irrespective of the levels of signal artifacts. %R 10.2196/17061 %U https://medinform.jmir.org/2020/2/e17061 %U https://doi.org/10.2196/17061 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 1 %P e15510 %T Longitudinal Risk Prediction of Chronic Kidney Disease in Diabetic Patients Using a Temporal-Enhanced Gradient Boosting Machine: Retrospective Cohort Study %A Song,Xing %A Waitman,Lemuel R %A Yu,Alan SL %A Robbins,David C %A Hu,Yong %A Liu,Mei %+ University of Kansas Medical Center, Department of Internal Medicine, Division of Medical Informatics, 3901 Rainbow Boulevard, Kansas City, KS, 66160, United States, 1 9139456446, meiliu@kumc.edu %K diabetic kidney disease %K diabetic nephropathy %K chronic kidney disease %K machine learning %D 2020 %7 31.1.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Artificial intelligence–enabled electronic health record (EHR) analysis can revolutionize medical practice from the diagnosis and prediction of complex diseases to making recommendations in patient care, especially for chronic conditions such as chronic kidney disease (CKD), which is one of the most frequent complications in patients with diabetes and is associated with substantial morbidity and mortality. Objective: The longitudinal prediction of health outcomes requires effective representation of temporal data in the EHR. In this study, we proposed a novel temporal-enhanced gradient boosting machine (GBM) model that dynamically updates and ensembles learners based on new events in patient timelines to improve the prediction accuracy of CKD among patients with diabetes. Methods: Using a broad spectrum of deidentified EHR data on a retrospective cohort of 14,039 adult patients with type 2 diabetes and GBM as the base learner, we validated our proposed Landmark-Boosting model against three state-of-the-art temporal models for rolling predictions of 1-year CKD risk. Results: The proposed model uniformly outperformed other models, achieving an area under receiver operating curve of 0.83 (95% CI 0.76-0.85), 0.78 (95% CI 0.75-0.82), and 0.82 (95% CI 0.78-0.86) in predicting CKD risk with automatic accumulation of new data in later years (years 2, 3, and 4 since diabetes mellitus onset, respectively). The Landmark-Boosting model also maintained the best calibration across moderate- and high-risk groups and over time. The experimental results demonstrated that the proposed temporal model can not only accurately predict 1-year CKD risk but also improve performance over time with additionally accumulated data, which is essential for clinical use to improve renal management of patients with diabetes. Conclusions: Incorporation of temporal information in EHR data can significantly improve predictive model performance and will particularly benefit patients who follow-up with their physicians as recommended. %M 32012067 %R 10.2196/15510 %U http://medinform.jmir.org/2020/1/e15510/ %U https://doi.org/10.2196/15510 %U http://www.ncbi.nlm.nih.gov/pubmed/32012067 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 1 %P e14679 %T Patient Perspectives on the Usefulness of an Artificial Intelligence–Assisted Symptom Checker: Cross-Sectional Survey Study %A Meyer,Ashley N D %A Giardina,Traber D %A Spitzmueller,Christiane %A Shahid,Umber %A Scott,Taylor M T %A Singh,Hardeep %+ Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey Veterans Affairs Medical Center and Baylor College of Medicine, 2002 Holcombe Blvd #152, Houston, TX, United States, 1 7134404660, ameyer@bcm.edu %K clinical decision support systems %K technology %K diagnosis %K patient safety %K symptom checker %K computer-assisted diagnosis %D 2020 %7 30.1.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Patients are increasingly seeking Web-based symptom checkers to obtain diagnoses. However, little is known about the characteristics of the patients who use these resources, their rationale for use, and whether they find them accurate and useful. Objective: The study aimed to examine patients’ experiences using an artificial intelligence (AI)–assisted online symptom checker. Methods: An online survey was administered between March 2, 2018, through March 15, 2018, to US users of the Isabel Symptom Checker within 6 months of their use. User characteristics, experiences of symptom checker use, experiences discussing results with physicians, and prior personal history of experiencing a diagnostic error were collected. Results: A total of 329 usable responses was obtained. The mean respondent age was 48.0 (SD 16.7) years; most were women (230/304, 75.7%) and white (271/304, 89.1%). Patients most commonly used the symptom checker to better understand the causes of their symptoms (232/304, 76.3%), followed by for deciding whether to seek care (101/304, 33.2%) or where (eg, primary or urgent care: 63/304, 20.7%), obtaining medical advice without going to a doctor (48/304, 15.8%), and understanding their diagnoses better (39/304, 12.8%). Most patients reported receiving useful information for their health problems (274/304, 90.1%), with half reporting positive health effects (154/302, 51.0%). Most patients perceived it to be useful as a diagnostic tool (253/301, 84.1%), as a tool providing insights leading them closer to correct diagnoses (231/303, 76.2%), and reported they would use it again (278/304, 91.4%). Patients who discussed findings with their physicians (103/213, 48.4%) more often felt physicians were interested (42/103, 40.8%) than not interested in learning about the tool’s results (24/103, 23.3%) and more often felt physicians were open (62/103, 60.2%) than not open (21/103, 20.4%) to discussing the results. Compared with patients who had not previously experienced diagnostic errors (missed or delayed diagnoses: 123/304, 40.5%), patients who had previously experienced diagnostic errors (181/304, 59.5%) were more likely to use the symptom checker to determine where they should seek care (15/123, 12.2% vs 48/181, 26.5%; P=.002), but they less often felt that physicians were interested in discussing the tool’s results (20/34, 59% vs 22/69, 32%; P=.04). Conclusions: Despite ongoing concerns about symptom checker accuracy, a large patient-user group perceived an AI-assisted symptom checker as useful for diagnosis. Formal validation studies evaluating symptom checker accuracy and effectiveness in real-world practice could provide additional useful information about their benefit. %M 32012052 %R 10.2196/14679 %U http://www.jmir.org/2020/1/e14679/ %U https://doi.org/10.2196/14679 %U http://www.ncbi.nlm.nih.gov/pubmed/32012052 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 1 %P e15645 %T The Detection of Opioid Misuse and Heroin Use From Paramedic Response Documentation: Machine Learning for Improved Surveillance %A Prieto,José Tomás %A Scott,Kenneth %A McEwen,Dean %A Podewils,Laura J %A Al-Tayyib,Alia %A Robinson,James %A Edwards,David %A Foldy,Seth %A Shlay,Judith C %A Davidson,Arthur J %+ Division of Scientific Education and Professional Development, Centers for Disease Control and Prevention, 1600 Clifton Rd, Atlanta, GA, 30333, United States, 1 3036024487, josetomasprieto@gmail.com %K naloxone %K emergency medical services %K natural language processing %K heroin %K substance-related disorders %K opioid crisis %K artificial intelligence %D 2020 %7 3.1.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Timely, precise, and localized surveillance of nonfatal events is needed to improve response and prevention of opioid-related problems in an evolving opioid crisis in the United States. Records of naloxone administration found in prehospital emergency medical services (EMS) data have helped estimate opioid overdose incidence, including nonhospital, field-treated cases. However, as naloxone is often used by EMS personnel in unconsciousness of unknown cause, attributing naloxone administration to opioid misuse and heroin use (OM) may misclassify events. Better methods are needed to identify OM. Objective: This study aimed to develop and test a natural language processing method that would improve identification of potential OM from paramedic documentation. Methods: First, we searched Denver Health paramedic trip reports from August 2017 to April 2018 for keywords naloxone, heroin, and both combined, and we reviewed narratives of identified reports to determine whether they constituted true cases of OM. Then, we used this human classification as reference standard and trained 4 machine learning models (random forest, k-nearest neighbors, support vector machines, and L1-regularized logistic regression). We selected the algorithm that produced the highest area under the receiver operating curve (AUC) for model assessment. Finally, we compared positive predictive value (PPV) of the highest performing machine learning algorithm with PPV of searches of keywords naloxone, heroin, and combination of both in the binary classification of OM in unseen September 2018 data. Results: In total, 54,359 trip reports were filed from August 2017 to April 2018. Approximately 1.09% (594/54,359) indicated naloxone administration. Among trip reports with reviewer agreement regarding OM in the narrative, 57.6% (292/516) were considered to include information revealing OM. Approximately 1.63% (884/54,359) of all trip reports mentioned heroin in the narrative. Among trip reports with reviewer agreement, 95.5% (784/821) were considered to include information revealing OM. Combined results accounted for 2.39% (1298/54,359) of trip reports. Among trip reports with reviewer agreement, 77.79% (907/1166) were considered to include information consistent with OM. The reference standard used to train and test machine learning models included details of 1166 trip reports. L1-regularized logistic regression was the highest performing algorithm (AUC=0.94; 95% CI 0.91-0.97) in identifying OM. Tested on 5983 unseen reports from September 2018, the keyword naloxone inaccurately identified and underestimated probable OM trip report cases (63 cases; PPV=0.68). The keyword heroin yielded more cases with improved performance (129 cases; PPV=0.99). Combined keyword and L1-regularized logistic regression classifier further improved performance (146 cases; PPV=0.99). Conclusions: A machine learning application enhanced the effectiveness of finding OM among documented paramedic field responses. This approach to refining OM surveillance may lead to improved first-responder and public health responses toward prevention of overdoses and other opioid-related problems in US communities. %M 31899451 %R 10.2196/15645 %U https://www.jmir.org/2020/1/e15645 %U https://doi.org/10.2196/15645 %U http://www.ncbi.nlm.nih.gov/pubmed/31899451 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 8 %N 1 %P e13244 %T Applicability of the User Engagement Scale to Mobile Health: A Survey-Based Quantitative Study %A Holdener,Marianne %A Gut,Alain %A Angerer,Alfred %+ Winterthur Institute of Health Economics, School of Management and Law, Zurich University of Applied Sciences, Gertrudstrasse 15, Winterthur, 8401, Switzerland, 41 798145158, marianneholdener@bluemail.ch %K mobile health %K mhealth %K mobile apps %K user engagement %K measurement %K user engagement scale %K chatbot %D 2020 %7 3.1.2020 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: There has recently been exponential growth in the development and use of health apps on mobile phones. As with most mobile apps, however, the majority of users abandon them quickly and after minimal use. One of the most critical factors for the success of a health app is how to support users’ commitment to their health. Despite increased interest from researchers in mobile health, few studies have examined the measurement of user engagement with health apps. Objective: User engagement is a multidimensional, complex phenomenon. The aim of this study was to understand the concept of user engagement and, in particular, to demonstrate the applicability of a user engagement scale (UES) to mobile health apps. Methods: To determine the measurability of user engagement in a mobile health context, a UES was employed, which is a psychometric tool to measure user engagement with a digital system. This was adapted to Ada, developed by Ada Health, an artificial intelligence–powered personalized health guide that helps people understand their health. A principal component analysis (PCA) with varimax rotation was conducted on 30 items. In addition, sum scores as means of each subscale were calculated. Results: Survey data from 73 Ada users were analyzed. PCA was determined to be suitable, as verified by the sampling adequacy of Kaiser-Meyer-Olkin=0.858, a significant Bartlett test of sphericity (χ2300=1127.1; P<.001), and communalities mostly within the 0.7 range. Although 5 items had to be removed because of low factor loadings, the results of the remaining 25 items revealed 4 attributes: perceived usability, aesthetic appeal, reward, and focused attention. Ada users showed the highest engagement level with perceived usability, with a value of 294, followed by aesthetic appeal, reward, and focused attention. Conclusions: Although the UES was deployed in German and adapted to another digital domain, PCA yielded consistent subscales and a 4-factor structure. This indicates that user engagement with health apps can be assessed with the German version of the UES. These results can benefit related mobile health app engagement research and may be of importance to marketers and app developers. %M 31899454 %R 10.2196/13244 %U https://mhealth.jmir.org/2020/1/e13244 %U https://doi.org/10.2196/13244 %U http://www.ncbi.nlm.nih.gov/pubmed/31899454 %0 Journal Article %@ 2561-7605 %I JMIR Publications %V 2 %N 2 %P e15381 %T Exploring Older Adults’ Beliefs About the Use of Intelligent Assistants for Consumer Health Information Management: A Participatory Design Study %A Martin-Hammond,Aqueasha %A Vemireddy,Sravani %A Rao,Kartik %+ Department of Human-Centered Computing, School of Informatics and Computing, Indiana University-Purdue University Indianapolis, 535 West Michigan St, Indianapolis, IN, 46202, United States, 1 3172787686, aqumarti@iupui.edu %K intelligent assistants %K artificial intelligence %K chatbots %K conversational agents %K digital health %K elderly %K aging in place %K participatory design %K co-design %K health information seeking %D 2019 %7 11.12.2019 %9 Original Paper %J JMIR Aging %G English %X Background: Intelligent assistants (IAs), also known as intelligent agents, use artificial intelligence to help users achieve a goal or complete a task. IAs represent a potential solution for providing older adults with individualized assistance at home, for example, to reduce social isolation, serve as memory aids, or help with disease management. However, to design IAs for health that are beneficial and accepted by older adults, it is important to understand their beliefs about IAs, how they would like to interact with IAs for consumer health, and how they desire to integrate IAs into their homes. Objective: We explore older adults’ mental models and beliefs about IAs, the tasks they want IAs to support, and how they would like to interact with IAs for consumer health. For the purpose of this study, we focus on IAs in the context of consumer health information management and search. Methods: We present findings from an exploratory, qualitative study that investigated older adults’ perspectives of IAs that aid with consumer health information search and management tasks. Eighteen older adults participated in a multiphase, participatory design workshop in which we engaged them in discussion, brainstorming, and design activities that helped us identify their current challenges managing and finding health information at home. We also explored their beliefs and ideas for an IA to assist them with consumer health tasks. We used participatory design activities to identify areas in which they felt IAs might be useful, but also to uncover the reasoning behind the ideas they presented. Discussions were audio-recorded and later transcribed. We compiled design artifacts collected during the study to supplement researcher transcripts and notes. Thematic analysis was used to analyze data. Results: We found that participants saw IAs as potentially useful for providing recommendations, facilitating collaboration between themselves and other caregivers, and for alerts of serious illness. However, they also desired familiar and natural interactions with IAs (eg, using voice) that could, if need be, provide fluid and unconstrained interactions, reason about their symptoms, and provide information or advice. Other participants discussed the need for flexible IAs that could be used by those with low technical resources or skills. Conclusions: From our findings, we present a discussion of three key components of participants’ mental models, including the people, behaviors, and interactions they described that were important for IAs for consumer health information management and seeking. We then discuss the role of access, transparency, caregivers, and autonomy in design for addressing participants’ concerns about privacy and trust as well as its role in assisting others that may interact with an IA on the older adults’ behalf. International Registered Report Identifier (IRRID): RR2-10.1145/3240925.3240972 %M 31825322 %R 10.2196/15381 %U http://aging.jmir.org/2019/2/e15381/ %U https://doi.org/10.2196/15381 %U http://www.ncbi.nlm.nih.gov/pubmed/31825322 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 4 %P e13430 %T Impact of Automatic Query Generation and Quality Recognition Using Deep Learning to Curate Evidence From Biomedical Literature: Empirical Study %A Afzal,Muhammad %A Hussain,Maqbool %A Malik,Khalid Mahmood %A Lee,Sungyoung %+ Department of Computer Science and Engineering, Kyung Hee University, Room 313, Yongin, 446-701, Republic of Korea, 82 312012514, sylee@oslab.khu.ac.kr %K data curation %K evidence-based medicine %K clinical decision support systems %K precision medicine %K biomedical research %K machine learning %K deep learning %D 2019 %7 9.12.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: The quality of health care is continuously improving and is expected to improve further because of the advancement of machine learning and knowledge-based techniques along with innovation and availability of wearable sensors. With these advancements, health care professionals are now becoming more interested and involved in seeking scientific research evidence from external sources for decision making relevant to medical diagnosis, treatments, and prognosis. Not much work has been done to develop methods for unobtrusive and seamless curation of data from the biomedical literature. Objective: This study aimed to design a framework that can enable bringing quality publications intelligently to the users’ desk to assist medical practitioners in answering clinical questions and fulfilling their informational needs. Methods: The proposed framework consists of methods for efficient biomedical literature curation, including the automatic construction of a well-built question, the recognition of evidence quality by proposing extended quality recognition model (E-QRM), and the ranking and summarization of the extracted evidence. Results: Unlike previous works, the proposed framework systematically integrates the echelons of biomedical literature curation by including methods for searching queries, content quality assessments, and ranking and summarization. Using an ensemble approach, our high-impact classifier E-QRM obtained significantly improved accuracy than the existing quality recognition model (1723/1894, 90.97% vs 1462/1894, 77.21%). Conclusions: Our proposed methods and evaluation demonstrate the validity and rigorousness of the results, which can be used in different applications, including evidence-based medicine, precision medicine, and medical education. %M 31815673 %R 10.2196/13430 %U http://medinform.jmir.org/2019/4/e13430/ %U https://doi.org/10.2196/13430 %U http://www.ncbi.nlm.nih.gov/pubmed/31815673 %0 Journal Article %@ 2369-3762 %I JMIR Publications %V 5 %N 2 %P e16048 %T Introducing Artificial Intelligence Training in Medical Education %A Paranjape,Ketan %A Schinkel,Michiel %A Nannan Panday,Rishi %A Car,Josip %A Nanayakkara,Prabath %+ Amsterdam University Medical Center, De Boelelaan 1117, 1081 HV, Amsterdam, Netherlands, 31 3174108035, ketanp@alumni.gsb.stanford.edu %K algorithm %K artificial intelligence %K black box %K deep learning %K machine learning %K medical education %K continuing education %K data sciences %K curriculum %D 2019 %7 3.12.2019 %9 Viewpoint %J JMIR Med Educ %G English %X Health care is evolving and with it the need to reform medical education. As the practice of medicine enters the age of artificial intelligence (AI), the use of data to improve clinical decision making will grow, pushing the need for skillful medicine-machine interaction. As the rate of medical knowledge grows, technologies such as AI are needed to enable health care professionals to effectively use this knowledge to practice medicine. Medical professionals need to be adequately trained in this new technology, its advantages to improve cost, quality, and access to health care, and its shortfalls such as transparency and liability. AI needs to be seamlessly integrated across different aspects of the curriculum. In this paper, we have addressed the state of medical education at present and have recommended a framework on how to evolve the medical education curriculum to include AI. %M 31793895 %R 10.2196/16048 %U http://mededu.jmir.org/2019/2/e16048/ %U https://doi.org/10.2196/16048 %U http://www.ncbi.nlm.nih.gov/pubmed/31793895 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 11 %P e15406 %T Artificial Intelligence Technologies for Coping with Alarm Fatigue in Hospital Environments Because of Sensory Overload: Algorithm Development and Validation %A Fernandes,Chrystinne Oliveira %A Miles,Simon %A Lucena,Carlos José Pereira De %A Cowan,Donald %+ Department of Informatics, Pontifical Catholic University of Rio de Janeiro, Rio Datacenter, 4th Fl, 225 Marquês de São Vicente St, Rio de Janeiro, 22451-900, Brazil, 55 21 3527 1510, chrystinne@gmail.com %K alert fatigue health personnel %K health information systems %K patient monitoring %K alert systems %K artificial intelligence %D 2019 %7 26.11.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Informed estimates claim that 80% to 99% of alarms set off in hospital units are false or clinically insignificant, representing a cacophony of sounds that do not present a real danger to patients. These false alarms can lead to an alert overload that causes a health care provider to miss important events that could be harmful or even life-threatening. As health care units become more dependent on monitoring devices for patient care purposes, the alarm fatigue issue has to be addressed as a major concern for the health care team as well as to enhance patient safety. Objective: The main goal of this paper was to propose a feasible solution for the alarm fatigue problem by using an automatic reasoning mechanism to decide how to notify members of the health care team. The aim was to reduce the number of notifications sent by determining whether or not to group a set of alarms that occur over a short period of time to deliver them together, without compromising patient safety. Methods: This paper describes: (1) a model for supporting reasoning algorithms that decide how to notify caregivers to avoid alarm fatigue; (2) an architecture for health systems that support patient monitoring and notification capabilities; and (3) a reasoning algorithm that specifies how to notify caregivers by deciding whether to aggregate a group of alarms to avoid alarm fatigue. Results: Experiments were used to demonstrate that providing a reasoning system can reduce the notifications received by the caregivers by up to 99.3% (582/586) of the total alarms generated. Our experiments were evaluated through the use of a dataset comprising patient monitoring data and vital signs recorded during 32 surgical cases where patients underwent anesthesia at the Royal Adelaide Hospital. We present the results of our algorithm by using graphs we generated using the R language, where we show whether the algorithm decided to deliver an alarm immediately or after a delay. Conclusions: The experimental results strongly suggest that this reasoning algorithm is a useful strategy for avoiding alarm fatigue. Although we evaluated our algorithm in an experimental environment, we tried to reproduce the context of a clinical environment by using real-world patient data. Our future work is to reproduce the evaluation study based on more realistic clinical conditions by increasing the number of patients, monitoring parameters, and types of alarm. %M 31769762 %R 10.2196/15406 %U http://www.jmir.org/2019/11/e15406/ %U https://doi.org/10.2196/15406 %U http://www.ncbi.nlm.nih.gov/pubmed/31769762 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 11 %P e16295 %T The Real Era of the Art of Medicine Begins with Artificial Intelligence %A Meskó,Bertalan %+ The Medical Futurist Institute, Povl Bang-Jensen u. 2/B1. 4/1, Budapest, 1118, Hungary, 36 703807260, berci@medicalfuturist.com %K future %K artificial intelligence %K digital health %K technology %K art of medicine %D 2019 %7 18.11.2019 %9 Viewpoint %J J Med Internet Res %G English %X Physicians have been performing the art of medicine for hundreds of years, and since the ancient era, patients have turned to physicians for help, advice, and cures. When the fathers of medicine started writing down their experience, knowledge, and observations, treating medical conditions became a structured process, with textbooks and professors sharing their methods over generations. After evidence-based medicine was established as the new form of medical science, the art and science of medicine had to be connected. As a result, by the end of the 20th century, health care had become highly dependent on technology. From electronic medical records, telemedicine, three-dimensional printing, algorithms, and sensors, technology has started to influence medical decisions and the lives of patients. While digital health technologies might be considered a threat to the art of medicine, I argue that advanced technologies, such as artificial intelligence, will initiate the real era of the art of medicine. Through the use of reinforcement learning, artificial intelligence could become the stethoscope of the 21st century. If we embrace these tools, the real art of medicine will begin now with the era of artificial intelligence. %M 31738169 %R 10.2196/16295 %U http://www.jmir.org/2019/11/e16295/ %U https://doi.org/10.2196/16295 %U http://www.ncbi.nlm.nih.gov/pubmed/31738169 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 8 %N 11 %P e14245 %T Real-Time Detection of Behavioral Anomalies of Older People Using Artificial Intelligence (The 3-PEGASE Study): Protocol for a Real-Life Prospective Trial %A Piau,Antoine %A Lepage,Benoit %A Bernon,Carole %A Gleizes,Marie-Pierre %A Nourhashemi,Fati %+ Gérontopôle, University Hospital of Toulouse, 4 Rue du Pont St Pierre, Toulouse, F-31400, France, 33 659561628, antoinepiau@hotmail.com %K frailty %K monitoring %K sensors %K artificial intelligence %K older adults %K participatory design %D 2019 %7 18.11.2019 %9 Protocol %J JMIR Res Protoc %G English %X Background: Most frail older persons are living at home, and we face difficulties in achieving seamless monitoring to detect adverse health changes. Even more important, this lack of follow-up could have a negative impact on the living choices made by older individuals and their care partners. People could give up their homes for the more reassuring environment of a medicalized living facility. We have developed a low-cost unobtrusive sensor-based solution to trigger automatic alerts in case of an acute event or subtle changes over time. It could facilitate older adults’ follow-up in their own homes, and thus support independent living. Objective: The primary objective of this prospective open-label study is to evaluate the relevance of the automatic alerts generated by our artificial intelligence–driven monitoring solution as judged by the recipients: older adults, caregivers, and professional support workers. The secondary objective is to evaluate its ability to detect subtle functional and cognitive decline and major medical events. Methods: The primary outcome will be evaluated for each successive 2-month follow-up period to estimate the progression of our learning algorithm performance over time. In total, 25 frail or disabled participants, aged 75 years and above and living alone in their own homes, will be enrolled for a 6-month follow-up period. Results: The first phase with 5 participants for a 4-month feasibility period has been completed and the expected completion date for the second phase of the study (20 participants for 6 months) is July 2020. Conclusions: The originality of our real-life project lies in the choice of the primary outcome and in our user-centered evaluation. We will evaluate the relevance of the alerts and the algorithm performance over time according to the end users. The first-line recipients of the information are the older adults and their care partners rather than health care professionals. Despite the fast pace of electronic health devices development, few studies have addressed the specific everyday needs of older adults and their families. Trial Registration: ClinicalTrials.gov NCT03484156; https://clinicaltrials.gov/ct2/show/NCT03484156 International Registered Report Identifier (IRRID): PRR1-10.2196/14245 %M 31738180 %R 10.2196/14245 %U http://www.researchprotocols.org/2019/11/e14245/ %U https://doi.org/10.2196/14245 %U http://www.ncbi.nlm.nih.gov/pubmed/31738180 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 11 %P e16607 %T Unlocking the Power of Artificial Intelligence and Big Data in Medicine %A Lovis,Christian %+ Division of Medical Information Sciences, University Hospitals of Geneva, Gabrielle Perret Gentil 4, Geneva, 1205, Switzerland, 41 22 37 26201, Christian.Lovis@hcuge.ch %K medical informatics %K artificial intelligence %K big data %D 2019 %7 8.11.2019 %9 Viewpoint %J J Med Internet Res %G English %X Data-driven science and its corollaries in machine learning and the wider field of artificial intelligence have the potential to drive important changes in medicine. However, medicine is not a science like any other: It is deeply and tightly bound with a large and wide network of legal, ethical, regulatory, economical, and societal dependencies. As a consequence, the scientific and technological progresses in handling information and its further processing and cross-linking for decision support and predictive systems must be accompanied by parallel changes in the global environment, with numerous stakeholders, including citizen and society. What can be seen at the first glance as a barrier and a mechanism slowing down the progression of data science must, however, be considered an important asset. Only global adoption can transform the potential of big data and artificial intelligence into an effective breakthroughs in handling health and medicine. This requires science and society, scientists and citizens, to progress together. %M 31702565 %R 10.2196/16607 %U https://www.jmir.org/2019/11/e16607 %U https://doi.org/10.2196/16607 %U http://www.ncbi.nlm.nih.gov/pubmed/31702565 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 11 %P e15360 %T The Personalization of Conversational Agents in Health Care: Systematic Review %A Kocaballi,Ahmet Baki %A Berkovsky,Shlomo %A Quiroz,Juan C %A Laranjo,Liliana %A Tong,Huong Ly %A Rezazadegan,Dana %A Briatore,Agustina %A Coiera,Enrico %+ Australian Institute of Health Innovation
, Faculty of Medicine and Health Sciences, Macquarie University, Level 6, 75 Talavera Road, Sydney, 2109, Australia, 61 298502465, baki.kocaballi@mq.edu.au %K conversational interfaces %K conversational agents %K dialogue systems %K personalization %K customization %K adaptive systems %K health care %D 2019 %7 7.11.2019 %9 Review %J J Med Internet Res %G English %X Background: The personalization of conversational agents with natural language user interfaces is seeing increasing use in health care applications, shaping the content, structure, or purpose of the dialogue between humans and conversational agents. Objective: The goal of this systematic review was to understand the ways in which personalization has been used with conversational agents in health care and characterize the methods of its implementation. Methods: We searched on PubMed, Embase, CINAHL, PsycInfo, and ACM Digital Library using a predefined search strategy. The studies were included if they: (1) were primary research studies that focused on consumers, caregivers, or health care professionals; (2) involved a conversational agent with an unconstrained natural language interface; (3) tested the system with human subjects; and (4) implemented personalization features. Results: The search found 1958 publications. After abstract and full-text screening, 13 studies were included in the review. Common examples of personalized content included feedback, daily health reports, alerts, warnings, and recommendations. The personalization features were implemented without a theoretical framework of customization and with limited evaluation of its impact. While conversational agents with personalization features were reported to improve user satisfaction, user engagement and dialogue quality, the role of personalization in improving health outcomes was not assessed directly. Conclusions: Most of the studies in our review implemented the personalization features without theoretical or evidence-based support for them and did not leverage the recent developments in other domains of personalization. Future research could incorporate personalization as a distinct design factor with a more careful consideration of its impact on health outcomes and its implications on patient safety, privacy, and decision-making. %M 31697237 %R 10.2196/15360 %U https://www.jmir.org/2019/11/e15360 %U https://doi.org/10.2196/15360 %U http://www.ncbi.nlm.nih.gov/pubmed/31697237 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 11 %P e15511 %T Modeling Research Topics for Artificial Intelligence Applications in Medicine: Latent Dirichlet Allocation Application Study %A Tran,Bach Xuan %A Nghiem,Son %A Sahin,Oz %A Vu,Tuan Manh %A Ha,Giang Hai %A Vu,Giang Thu %A Pham,Hai Quang %A Do,Hoa Thi %A Latkin,Carl A %A Tam,Wilson %A Ho,Cyrus S H %A Ho,Roger C M %+ Institute for Preventive Medicine and Public Health, Hanoi Medical University, No 1 Ton That Tung Street, Hanoi, 100000, Vietnam, 84 98 222 8662, bach.ipmph@gmail.com %K artificial intelligence %K applications %K medicine %K scientometric %K bibliometric %K latent Dirichlet allocation %D 2019 %7 1.11.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Artificial intelligence (AI)–based technologies develop rapidly and have myriad applications in medicine and health care. However, there is a lack of comprehensive reporting on the productivity, workflow, topics, and research landscape of AI in this field. Objective: This study aimed to evaluate the global development of scientific publications and constructed interdisciplinary research topics on the theory and practice of AI in medicine from 1977 to 2018. Methods: We obtained bibliographic data and abstract contents of publications published between 1977 and 2018 from the Web of Science database. A total of 27,451 eligible articles were analyzed. Research topics were classified by latent Dirichlet allocation, and principal component analysis was used to identify the construct of the research landscape. Results: The applications of AI have mainly impacted clinical settings (enhanced prognosis and diagnosis, robot-assisted surgery, and rehabilitation), data science and precision medicine (collecting individual data for precision medicine), and policy making (raising ethical and legal issues, especially regarding privacy and confidentiality of data). However, AI applications have not been commonly used in resource-poor settings due to the limit in infrastructure and human resources. Conclusions: The application of AI in medicine has grown rapidly and focuses on three leading platforms: clinical practices, clinical material, and policies. AI might be one of the methods to narrow down the inequality in health care and medicine between developing and developed countries. Technology transfer and support from developed countries are essential measures for the advancement of AI application in health care in developing countries. %M 31682577 %R 10.2196/15511 %U https://www.jmir.org/2019/11/e15511 %U https://doi.org/10.2196/15511 %U http://www.ncbi.nlm.nih.gov/pubmed/31682577 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 7 %N 11 %P e14452 %T Development of a Deep Learning Model for Dynamic Forecasting of Blood Glucose Level for Type 2 Diabetes Mellitus: Secondary Analysis of a Randomized Controlled Trial %A Faruqui,Syed Hasib Akhter %A Du,Yan %A Meka,Rajitha %A Alaeddini,Adel %A Li,Chengdong %A Shirinkam,Sara %A Wang,Jing %+ Center on Smart and Connected Health Technologies, University of Texas Health Science Center at San Antonio, 7703 Floyd Curl Drive, San Antonio, TX, United States, 1 210 450 8561, wangj1@uthscsa.edu %K type 2 diabetes %K long short-term memory (LSTM)-based recurrent neural networks (RNNs) %K glucose level prediction %K mobile health lifestyle data %D 2019 %7 1.11.2019 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: Type 2 diabetes mellitus (T2DM) is a major public health burden. Self-management of diabetes including maintaining a healthy lifestyle is essential for glycemic control and to prevent diabetes complications. Mobile-based health data can play an important role in the forecasting of blood glucose levels for lifestyle management and control of T2DM. Objective: The objective of this work was to dynamically forecast daily glucose levels in patients with T2DM based on their daily mobile health lifestyle data including diet, physical activity, weight, and glucose level from the day before. Methods: We used data from 10 T2DM patients who were overweight or obese in a behavioral lifestyle intervention using mobile tools for daily monitoring of diet, physical activity, weight, and blood glucose over 6 months. We developed a deep learning model based on long short-term memory–based recurrent neural networks to forecast the next-day glucose levels in individual patients. The neural network used several layers of computational nodes to model how mobile health data (food intake including consumed calories, fat, and carbohydrates; exercise; and weight) were progressing from one day to another from noisy data. Results: The model was validated based on a data set of 10 patients who had been monitored daily for over 6 months. The proposed deep learning model demonstrated considerable accuracy in predicting the next day glucose level based on Clark Error Grid and ±10% range of the actual values. Conclusions: Using machine learning methodologies may leverage mobile health lifestyle data to develop effective individualized prediction plans for T2DM management. However, predicting future glucose levels is challenging as glucose level is determined by multiple factors. Future study with more rigorous study design is warranted to better predict future glucose levels for T2DM management. %M 31682586 %R 10.2196/14452 %U https://mhealth.jmir.org/2019/11/e14452 %U https://doi.org/10.2196/14452 %U http://www.ncbi.nlm.nih.gov/pubmed/31682586 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 4 %P e15980 %T Cohort Selection for Clinical Trials From Longitudinal Patient Records: Text Mining Approach %A Spasic,Irena %A Krzeminski,Dominik %A Corcoran,Padraig %A Balinsky,Alexander %+ School of Computer Science & Informatics, Cardiff University, 5 The Parade, Cardiff, CF24 3AA, United Kingdom, 44 02920870320, spasici@cardiff.ac.uk %K natural language processing %K machine learning %K electronic medical records %K clinical trial %K eligibility determination %D 2019 %7 31.10.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: Clinical trials are an important step in introducing new interventions into clinical practice by generating data on their safety and efficacy. Clinical trials need to ensure that participants are similar so that the findings can be attributed to the interventions studied and not to some other factors. Therefore, each clinical trial defines eligibility criteria, which describe characteristics that must be shared by the participants. Unfortunately, the complexities of eligibility criteria may not allow them to be translated directly into readily executable database queries. Instead, they may require careful analysis of the narrative sections of medical records. Manual screening of medical records is time consuming, thus negatively affecting the timeliness of the recruitment process. Objective: Track 1 of the 2018 National Natural Language Processing Clinical Challenge focused on the task of cohort selection for clinical trials, aiming to answer the following question: Can natural language processing be applied to narrative medical records to identify patients who meet eligibility criteria for clinical trials? The task required the participating systems to analyze longitudinal patient records to determine if the corresponding patients met the given eligibility criteria. We aimed to describe a system developed to address this task. Methods: Our system consisted of 13 classifiers, one for each eligibility criterion. All classifiers used a bag-of-words document representation model. To prevent the loss of relevant contextual information associated with such representation, a pattern-matching approach was used to extract context-sensitive features. They were embedded back into the text as lexically distinguishable tokens, which were consequently featured in the bag-of-words representation. Supervised machine learning was chosen wherever a sufficient number of both positive and negative instances was available to learn from. A rule-based approach focusing on a small set of relevant features was chosen for the remaining criteria. Results: The system was evaluated using microaveraged F measure. Overall, 4 machine algorithms, including support vector machine, logistic regression, naïve Bayesian classifier, and gradient tree boosting (GTB), were evaluated on the training data using 10–fold cross-validation. Overall, GTB demonstrated the most consistent performance. Its performance peaked when oversampling was used to balance the training data. The final evaluation was performed on previously unseen test data. On average, the F measure of 89.04% was comparable to 3 of the top ranked performances in the shared task (91.11%, 90.28%, and 90.21%). With an F measure of 88.14%, we significantly outperformed these systems (81.03%, 78.50%, and 70.81%) in identifying patients with advanced coronary artery disease. Conclusions: The holdout evaluation provides evidence that our system was able to identify eligible patients for the given clinical trial with high accuracy. Our approach demonstrates how rule-based knowledge infusion can improve the performance of machine learning algorithms even when trained on a relatively small dataset. %M 31674914 %R 10.2196/15980 %U http://medinform.jmir.org/2019/4/e15980/ %U https://doi.org/10.2196/15980 %U http://www.ncbi.nlm.nih.gov/pubmed/31674914 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 10 %P e16222 %T Trust Me, I’m a Chatbot: How Artificial Intelligence in Health Care Fails the Turing Test %A Powell,John %+ Nuffield Department of Primary Care Health Sciences, Medical Sciences Division, University of Oxford, Radcliffe Observatory Quarter, 43 Woodstock Road, Oxford, OX2 6GG, United Kingdom, 44 1865617768 ext 617768, john.powell@phc.ox.ac.uk %K artificial intelligence %K machine learning %K medical informatics %K digital health %K ehealth %K chatbots %K conversational agents %D 2019 %7 28.10.2019 %9 Viewpoint %J J Med Internet Res %G English %X Over the next decade, one issue which will dominate sociotechnical studies in health informatics is the extent to which the promise of artificial intelligence in health care will be realized, along with the social and ethical issues which accompany it. A useful thought experiment is the application of the Turing test to user-facing artificial intelligence systems in health care (such as chatbots or conversational agents). In this paper I argue that many medical decisions require value judgements and the doctor-patient relationship requires empathy and understanding to arrive at a shared decision, often handling large areas of uncertainty and balancing competing risks. Arguably, medicine requires wisdom more than intelligence, artificial or otherwise. Artificial intelligence therefore needs to supplement rather than replace medical professionals, and identifying the complementary positioning of artificial intelligence in medical consultation is a key challenge for the future. In health care, artificial intelligence needs to pass the implementation game, not the imitation game. %M 31661083 %R 10.2196/16222 %U http://www.jmir.org/2019/10/e16222/ %U https://doi.org/10.2196/16222 %U http://www.ncbi.nlm.nih.gov/pubmed/31661083 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 6 %N 10 %P e14166 %T Conversational Agents in the Treatment of Mental Health Problems: Mixed-Method Systematic Review %A Gaffney,Hannah %A Mansell,Warren %A Tai,Sara %+ , School of Health Sciences, Faculty of Biology, Medicine and Health, University of Manchester, 2nd Floor, Zochonis Building, Manchester, M13 9PL, United Kingdom, 44 161 306 0400, hannah.gaffney-2@postgrad.manchester.ac.uk %K artificial intelligence %K mental health %K stress, pychological %K psychiatry %K therapy, computer-assisted %K conversational agent %K chatbot %K digital health %D 2019 %7 18.10.2019 %9 Review %J JMIR Ment Health %G English %X Background: The use of conversational agent interventions (including chatbots and robots) in mental health is growing at a fast pace. Recent existing reviews have focused exclusively on a subset of embodied conversational agent interventions despite other modalities aiming to achieve the common goal of improved mental health. Objective: This study aimed to review the use of conversational agent interventions in the treatment of mental health problems. Methods: We performed a systematic search using relevant databases (MEDLINE, EMBASE, PsycINFO, Web of Science, and Cochrane library). Studies that reported on an autonomous conversational agent that simulated conversation and reported on a mental health outcome were included. Results: A total of 13 studies were included in the review. Among them, 4 full-scale randomized controlled trials (RCTs) were included. The rest were feasibility, pilot RCTs and quasi-experimental studies. Interventions were diverse in design and targeted a range of mental health problems using a wide variety of therapeutic orientations. All included studies reported reductions in psychological distress postintervention. Furthermore, 5 controlled studies demonstrated significant reductions in psychological distress compared with inactive control groups. In addition, 3 controlled studies comparing interventions with active control groups failed to demonstrate superior effects. Broader utility in promoting well-being in nonclinical populations was unclear. Conclusions: The efficacy and acceptability of conversational agent interventions for mental health problems are promising. However, a more robust experimental design is required to demonstrate efficacy and efficiency. A focus on streamlining interventions, demonstrating equivalence to other treatment modalities, and elucidating mechanisms of action has the potential to increase acceptance by users and clinicians and maximize reach. %M 31628789 %R 10.2196/14166 %U https://mental.jmir.org/2019/10/e14166 %U https://doi.org/10.2196/14166 %U http://www.ncbi.nlm.nih.gov/pubmed/31628789 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 10 %P e14316 %T Psychosocial Factors Affecting Artificial Intelligence Adoption in Health Care in China: Cross-Sectional Study %A Ye,Tiantian %A Xue,Jiaolong %A He,Mingguang %A Gu,Jing %A Lin,Haotian %A Xu,Bin %A Cheng,Yu %+ Department of Medical Humanities, The Seventh Affiliated Hospital, Sun Yat-sen University, No 628, Zhenyuan Rd Guangming (New) Dist, Shenzhen, 518107, China, 86 02084114275, chengyu@mail.sysu.edu.cn %K artificial intelligence %K adoption %K technology acceptance model %K structural equation model %K intention %K subjective norms %K trust %K moderation %D 2019 %7 17.10.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Poor quality primary health care is a major issue in China, particularly in blindness prevention. Artificial intelligence (AI) could provide early screening and accurate auxiliary diagnosis to improve primary care services and reduce unnecessary referrals, but the application of AI in medical settings is still an emerging field. Objective: This study aimed to investigate the general public’s acceptance of ophthalmic AI devices, with reference to those already used in China, and the interrelated influencing factors that shape people’s intention to use these devices. Methods: We proposed a model of ophthalmic AI acceptance based on technology acceptance theories and variables from other health care–related studies. The model was verified via a 32-item questionnaire with 7-point Likert scales completed by 474 respondents (nationally random sampled). Structural equation modeling was used to evaluate item and construct reliability and validity via a confirmatory factor analysis, and the model’s path effects, significance, goodness of fit, and mediation and moderation effects were analyzed. Results: Standardized factor loadings of items were between 0.583 and 0.876. Composite reliability of 9 constructs ranged from 0.673 to 0.841. The discriminant validity of all constructs met the Fornell and Larcker criteria. Model fit indicators such as standardized root mean square residual (0.057), comparative fit index (0.915), and root mean squared error of approximation (0.049) demonstrated good fit. Intention to use (R2=0.515) is significantly affected by subjective norms (beta=.408; P<.001), perceived usefulness (beta=.336; P=.03), and resistance bias (beta=–.237; P=.02). Subjective norms and perceived behavior control had an indirect impact on intention to use through perceived usefulness and perceived ease of use. Eye health consciousness had an indirect positive effect on intention to use through perceived usefulness. Trust had a significant moderation effect (beta=–.095; P=.049) on the effect path of perceived usefulness to intention to use. Conclusions: The item, construct, and model indicators indicate reliable interpretation power and help explain the levels of public acceptance of ophthalmic AI devices in China. The influence of subjective norms can be linked to Confucian culture, collectivism, authoritarianism, and conformity mentality in China. Overall, the use of AI in diagnostics and clinical laboratory analysis is underdeveloped, and the Chinese public are generally mistrustful of medical staff and the Chinese medical system. Stakeholders such as doctors and AI suppliers should therefore avoid making misleading or over-exaggerated claims in the promotion of AI health care products. %M 31625950 %R 10.2196/14316 %U http://www.jmir.org/2019/10/e14316/ %U https://doi.org/10.2196/14316 %U http://www.ncbi.nlm.nih.gov/pubmed/31625950 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 4 %P e14806 %T A Deep Learning Approach for Managing Medical Consumable Materials in Intensive Care Units via Convolutional Neural Networks: Technical Proof-of-Concept Study %A Peine,Arne %A Hallawa,Ahmed %A Schöffski,Oliver %A Dartmann,Guido %A Fazlic,Lejla Begic %A Schmeink,Anke %A Marx,Gernot %A Martin,Lukas %+ Department of Intensive Care Medicine and Intermediate Care, University Hospital Rheinisch-Westfälische Technische Hochschule Aachen, Pauwelsstr 30, Aachen, 52074, Germany, 49 241 800, apeine@ukaachen.de %K convolutional neural networks %K deep learning, critical care %K intensive care %K image recognition %K medical economics %K medical consumables %K artificial intelligence %K machine learning %D 2019 %7 10.10.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: High numbers of consumable medical materials (eg, sterile needles and swabs) are used during the daily routine of intensive care units (ICUs) worldwide. Although medical consumables largely contribute to total ICU hospital expenditure, many hospitals do not track the individual use of materials. Current tracking solutions meeting the specific requirements of the medical environment, like barcodes or radio frequency identification, require specialized material preparation and high infrastructure investment. This impedes the accurate prediction of consumption, leads to high storage maintenance costs caused by large inventories, and hinders scientific work due to inaccurate documentation. Thus, new cost-effective and contactless methods for object detection are urgently needed. Objective: The goal of this work was to develop and evaluate a contactless visual recognition system for tracking medical consumable materials in ICUs using a deep learning approach on a distributed client-server architecture. Methods: We developed Consumabot, a novel client-server optical recognition system for medical consumables, based on the convolutional neural network model MobileNet implemented in Tensorflow. The software was designed to run on single-board computer platforms as a detection unit. The system was trained to recognize 20 different materials in the ICU, while 100 sample images of each consumable material were provided. We assessed the top-1 recognition rates in the context of different real-world ICU settings: materials presented to the system without visual obstruction, 50% covered materials, and scenarios of multiple items. We further performed an analysis of variance with repeated measures to quantify the effect of adverse real-world circumstances. Results: Consumabot reached a >99% reliability of recognition after about 60 steps of training and 150 steps of validation. A desirable low cross entropy of <0.03 was reached for the training set after about 100 iteration steps and after 170 steps for the validation set. The system showed a high top-1 mean recognition accuracy in a real-world scenario of 0.85 (SD 0.11) for objects presented to the system without visual obstruction. Recognition accuracy was lower, but still acceptable, in scenarios where the objects were 50% covered (P<.001; mean recognition accuracy 0.71; SD 0.13) or multiple objects of the target group were present (P=.01; mean recognition accuracy 0.78; SD 0.11), compared to a nonobstructed view. The approach met the criteria of absence of explicit labeling (eg, barcodes, radio frequency labeling) while maintaining a high standard for quality and hygiene with minimal consumption of resources (eg, cost, time, training, and computational power). Conclusions: Using a convolutional neural network architecture, Consumabot consistently achieved good results in the classification of consumables and thus is a feasible way to recognize and register medical consumables directly to a hospital’s electronic health record. The system shows limitations when the materials are partially covered, therefore identifying characteristics of the consumables are not presented to the system. Further development of the assessment in different medical circumstances is needed. %M 31603430 %R 10.2196/14806 %U http://medinform.jmir.org/2019/4/e14806/ %U https://doi.org/10.2196/14806 %U http://www.ncbi.nlm.nih.gov/pubmed/31603430 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 4 %P e14401 %T Characterizing Artificial Intelligence Applications in Cancer Research: A Latent Dirichlet Allocation Analysis %A Tran,Bach Xuan %A Latkin,Carl A %A Sharafeldin,Noha %A Nguyen,Katherina %A Vu,Giang Thu %A Tam,Wilson W S %A Cheung,Ngai-Man %A Nguyen,Huong Lan Thi %A Ho,Cyrus S H %A Ho,Roger C M %+ Institute for Preventive Medicine and Public Health, Hanoi Medical University, No 1 Ton That Tung Street, Hanoi, 100000, Vietnam, 84 982228662, bach.ipmph@gmail.com %K scientometrics %K cancer %K artificial intelligence %K global %K mapping %D 2019 %7 15.9.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: Artificial intelligence (AI)–based therapeutics, devices, and systems are vital innovations in cancer control; particularly, they allow for diagnosis, screening, precise estimation of survival, informing therapy selection, and scaling up treatment services in a timely manner. Objective: The aim of this study was to analyze the global trends, patterns, and development of interdisciplinary landscapes in AI and cancer research. Methods: An exploratory factor analysis was conducted to identify research domains emerging from abstract contents. The Jaccard similarity index was utilized to identify the most frequently co-occurring terms. Latent Dirichlet Allocation was used for classifying papers into corresponding topics. Results: From 1991 to 2018, the number of studies examining the application of AI in cancer care has grown to 3555 papers covering therapeutics, capacities, and factors associated with outcomes. Topics with the highest volume of publications include (1) machine learning, (2) comparative effectiveness evaluation of AI-assisted medical therapies, and (3) AI-based prediction. Noticeably, this classification has revealed topics examining the incremental effectiveness of AI applications, the quality of life, and functioning of patients receiving these innovations. The growing research productivity and expansion of multidisciplinary approaches are largely driven by machine learning, artificial neural networks, and AI in various clinical practices. Conclusions: The research landscapes show that the development of AI in cancer care is focused on not only improving prediction in cancer screening and AI-assisted therapeutics but also on improving other corresponding areas such as precision and personalized medicine and patient-reported outcomes. %M 31573929 %R 10.2196/14401 %U https://medinform.jmir.org/2019/4/e14401 %U https://doi.org/10.2196/14401 %U http://www.ncbi.nlm.nih.gov/pubmed/31573929 %0 Journal Article %@ 2369-1999 %I JMIR Publications %V 5 %N 2 %P e12163 %T Developing Machine Learning Algorithms for the Prediction of Early Death in Elderly Cancer Patients: Usability Study %A Sena,Gabrielle Ribeiro %A Lima,Tiago Pessoa Ferreira %A Mello,Maria Julia Gonçalves %A Thuler,Luiz Claudio Santos %A Lima,Jurema Telles Oliveira %+ Department of Geriatric Oncology, Instituto de Medicina Integral Prof Fernando Figueira, Rua dos Coelhos 300, Recife, 50070-902, Brazil, 55 81 21224100, gabriellesena8@gmail.com %K geriatric assessment %K aged %K machine learning %K medical oncology %K death %D 2019 %7 26.9.2019 %9 Original Paper %J JMIR Cancer %G English %X Background: The importance of classifying cancer patients into high- or low-risk groups has led many research teams, from the biomedical and bioinformatics fields, to study the application of machine learning (ML) algorithms. The International Society of Geriatric Oncology recommends the use of the comprehensive geriatric assessment (CGA), a multidisciplinary tool to evaluate health domains, for the follow-up of elderly cancer patients. However, no applications of ML have been proposed using CGA to classify elderly cancer patients. Objective: The aim of this study was to propose and develop predictive models, using ML and CGA, to estimate the risk of early death in elderly cancer patients. Methods: The ability of ML algorithms to predict early mortality in a cohort involving 608 elderly cancer patients was evaluated. The CGA was conducted during admission by a multidisciplinary team and included the following questionnaires: mini-mental state examination (MMSE), geriatric depression scale-short form, international physical activity questionnaire-short form, timed up and go, Katz index of independence in activities of daily living, Charlson comorbidity index, Karnofsky performance scale (KPS), polypharmacy, and mini nutritional assessment-short form (MNA-SF). The 10-fold cross-validation algorithm was used to evaluate all possible combinations of these questionnaires to estimate the risk of early death, considered when occurring within 6 months of diagnosis, in a variety of ML classifiers, including Naive Bayes (NB), decision tree algorithm J48 (J48), and multilayer perceptron (MLP). On each fold of evaluation, tiebreaking is handled by choosing the smallest set of questionnaires. Results: It was possible to select CGA questionnaire subsets with high predictive capacity for early death, which were either statistically similar (NB) or higher (J48 and MLP) when compared with the use of all questionnaires investigated. These results show that CGA questionnaire selection can improve accuracy rates and decrease the time spent to evaluate elderly cancer patients. Conclusions: A simplified predictive model aiming to estimate the risk of early death in elderly cancer patients is proposed herein, minimally composed by the MNA-SF and KPS. We strongly recommend that these questionnaires be incorporated into regular geriatric assessment of older patients with cancer. %M 31573896 %R 10.2196/12163 %U https://cancer.jmir.org/2019/2/e12163 %U https://doi.org/10.2196/12163 %U http://www.ncbi.nlm.nih.gov/pubmed/31573896 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 3 %P e14830 %T Fine-Tuning Bidirectional Encoder Representations From Transformers (BERT)–Based Models on Large-Scale Electronic Health Record Notes: An Empirical Study %A Li,Fei %A Jin,Yonghao %A Liu,Weisong %A Rawat,Bhanu Pratap Singh %A Cai,Pengshan %A Yu,Hong %+ Department of Computer Science, University of Massachusetts Lowell, 1 University Avenue, Lowell, MA,, United States, 1 978 934 6132, Hong_Yu@uml.edu %K natural language processing %K entity normalization %K deep learning %K electronic health record note %K BERT %D 2019 %7 12.09.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: The bidirectional encoder representations from transformers (BERT) model has achieved great success in many natural language processing (NLP) tasks, such as named entity recognition and question answering. However, little prior work has explored this model to be used for an important task in the biomedical and clinical domains, namely entity normalization. Objective: We aim to investigate the effectiveness of BERT-based models for biomedical or clinical entity normalization. In addition, our second objective is to investigate whether the domains of training data influence the performances of BERT-based models as well as the degree of influence. Methods: Our data was comprised of 1.5 million unlabeled electronic health record (EHR) notes. We first fine-tuned BioBERT on this large collection of unlabeled EHR notes. This generated our BERT-based model trained using 1.5 million electronic health record notes (EhrBERT). We then further fine-tuned EhrBERT, BioBERT, and BERT on three annotated corpora for biomedical and clinical entity normalization: the Medication, Indication, and Adverse Drug Events (MADE) 1.0 corpus, the National Center for Biotechnology Information (NCBI) disease corpus, and the Chemical-Disease Relations (CDR) corpus. We compared our models with two state-of-the-art normalization systems, namely MetaMap and disease name normalization (DNorm). Results: EhrBERT achieved 40.95% F1 in the MADE 1.0 corpus for mapping named entities to the Medical Dictionary for Regulatory Activities and the Systematized Nomenclature of Medicine—Clinical Terms (SNOMED-CT), which have about 380,000 terms. In this corpus, EhrBERT outperformed MetaMap by 2.36% in F1. For the NCBI disease corpus and CDR corpus, EhrBERT also outperformed DNorm by improving the F1 scores from 88.37% and 89.92% to 90.35% and 93.82%, respectively. Compared with BioBERT and BERT, EhrBERT outperformed them on the MADE 1.0 corpus and the CDR corpus. Conclusions: Our work shows that BERT-based models have achieved state-of-the-art performance for biomedical and clinical entity normalization. BERT-based models can be readily fine-tuned to normalize any kind of named entities. %M 31516126 %R 10.2196/14830 %U http://medinform.jmir.org/2019/3/e14830/ %U https://doi.org/10.2196/14830 %U http://www.ncbi.nlm.nih.gov/pubmed/31516126 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 7 %N 8 %P e11966 %T Deep Learning Intervention for Health Care Challenges: Some Biomedical Domain Considerations %A Tobore,Igbe %A Li,Jingzhen %A Yuhang,Liu %A Al-Handarish,Yousef %A Kandwal,Abhishek %A Nie,Zedong %A Wang,Lei %+ Center for Medical Robotics and Minimally Invasive Surgical Devices, Shenzhen Institutes of Advance Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, Shenzhen University, Xili Town, Nanshan District, Shenzhen,, China, 86 755 86585213, zd.nie@siat.ac.cn %K machine learning %K deep learning %K big data %K mHealth %K medical imaging %K electronic health record %K biologicals %K biomedical %K ECG %K EEG %K artificial intelligence %D 2019 %7 02.08.2019 %9 Viewpoint %J JMIR Mhealth Uhealth %G English %X The use of deep learning (DL) for the analysis and diagnosis of biomedical and health care problems has received unprecedented attention in the last decade. The technique has recorded a number of achievements for unearthing meaningful features and accomplishing tasks that were hitherto difficult to solve by other methods and human experts. Currently, biological and medical devices, treatment, and applications are capable of generating large volumes of data in the form of images, sounds, text, graphs, and signals creating the concept of big data. The innovation of DL is a developing trend in the wake of big data for data representation and analysis. DL is a type of machine learning algorithm that has deeper (or more) hidden layers of similar function cascaded into the network and has the capability to make meaning from medical big data. Current transformation drivers to achieve personalized health care delivery will be possible with the use of mobile health (mHealth). DL can provide the analysis for the deluge of data generated from mHealth apps. This paper reviews the fundamentals of DL methods and presents a general view of the trends in DL by capturing literature from PubMed and the Institute of Electrical and Electronics Engineers database publications that implement different variants of DL. We highlight the implementation of DL in health care, which we categorize into biological system, electronic health record, medical image, and physiological signals. In addition, we discuss some inherent challenges of DL affecting biomedical and health domain, as well as prospective research directions that focus on improving health management by promoting the application of physiological signals and modern internet technology. %M 31376272 %R 10.2196/11966 %U https://mhealth.jmir.org/2019/8/e11966/ %U https://doi.org/10.2196/11966 %U http://www.ncbi.nlm.nih.gov/pubmed/31376272 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 3 %P e14499 %T Projection Word Embedding Model With Hybrid Sampling Training for Classifying ICD-10-CM Codes: Longitudinal Observational Study %A Lin,Chin %A Lou,Yu-Sheng %A Tsai,Dung-Jang %A Lee,Chia-Cheng %A Hsu,Chia-Jung %A Wu,Ding-Chung %A Wang,Mei-Chuen %A Fang,Wen-Hui %+ Department of Family and Community Medicine, Tri-Service General Hospital, National Defense Medical Center, No. 325, Section 2, Chenggong Road, Neihu District, Taipei, 11490, Taiwan, 886 02 87923100 ext 18448, rumaf.fang@gmail.com %K word embedding %K convolutional neural network %K artificial intelligence %K natural language processing %K electronic health records %D 2019 %7 23.7.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: Most current state-of-the-art models for searching the International Classification of Diseases, Tenth Revision Clinical Modification (ICD-10-CM) codes use word embedding technology to capture useful semantic properties. However, they are limited by the quality of initial word embeddings. Word embedding trained by electronic health records (EHRs) is considered the best, but the vocabulary diversity is limited by previous medical records. Thus, we require a word embedding model that maintains the vocabulary diversity of open internet databases and the medical terminology understanding of EHRs. Moreover, we need to consider the particularity of the disease classification, wherein discharge notes present only positive disease descriptions. Objective: We aimed to propose a projection word2vec model and a hybrid sampling method. In addition, we aimed to conduct a series of experiments to validate the effectiveness of these methods. Methods: We compared the projection word2vec model and traditional word2vec model using two corpora sources: English Wikipedia and PubMed journal abstracts. We used seven published datasets to measure the medical semantic understanding of the word2vec models and used these embeddings to identify the three–character-level ICD-10-CM diagnostic codes in a set of discharge notes. On the basis of embedding technology improvement, we also tried to apply the hybrid sampling method to improve accuracy. The 94,483 labeled discharge notes from the Tri-Service General Hospital of Taipei, Taiwan, from June 1, 2015, to June 30, 2017, were used. To evaluate the model performance, 24,762 discharge notes from July 1, 2017, to December 31, 2017, from the same hospital were used. Moreover, 74,324 additional discharge notes collected from seven other hospitals were tested. The F-measure, which is the major global measure of effectiveness, was adopted. Results: In medical semantic understanding, the original EHR embeddings and PubMed embeddings exhibited superior performance to the original Wikipedia embeddings. After projection training technology was applied, the projection Wikipedia embeddings exhibited an obvious improvement but did not reach the level of original EHR embeddings or PubMed embeddings. In the subsequent ICD-10-CM coding experiment, the model that used both projection PubMed and Wikipedia embeddings had the highest testing mean F-measure (0.7362 and 0.6693 in Tri-Service General Hospital and the seven other hospitals, respectively). Moreover, the hybrid sampling method was found to improve the model performance (F-measure=0.7371/0.6698). Conclusions: The word embeddings trained using EHR and PubMed could understand medical semantics better, and the proposed projection word2vec model improved the ability of medical semantics extraction in Wikipedia embeddings. Although the improvement from the projection word2vec model in the real ICD-10-CM coding task was not substantial, the models could effectively handle emerging diseases. The proposed hybrid sampling method enables the model to behave like a human expert. %R 10.2196/14499 %U http://medinform.jmir.org/2019/3/e14499/ %U https://doi.org/10.2196/14499 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 7 %P e13659 %T Artificial Intelligence and the Implementation Challenge %A Shaw,James %A Rudzicz,Frank %A Jamieson,Trevor %A Goldfarb,Avi %+ Women's College Hospital, Institute for Health System Solutions and Virtual Care, 76 Grenville Street, Toronto, ON, M5G2A2, Canada, 1 4163236400, jay.shaw@wchospital.ca %K artificial intelligence %K machine learning %K implementation science %K ethics %D 2019 %7 10.07.2019 %9 Viewpoint %J J Med Internet Res %G English %X Background: Applications of artificial intelligence (AI) in health care have garnered much attention in recent years, but the implementation issues posed by AI have not been substantially addressed. Objective: In this paper, we have focused on machine learning (ML) as a form of AI and have provided a framework for thinking about use cases of ML in health care. We have structured our discussion of challenges in the implementation of ML in comparison with other technologies using the framework of Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies (NASSS). Methods: After providing an overview of AI technology, we describe use cases of ML as falling into the categories of decision support and automation. We suggest these use cases apply to clinical, operational, and epidemiological tasks and that the primary function of ML in health care in the near term will be decision support. We then outline unique implementation issues posed by ML initiatives in the categories addressed by the NASSS framework, specifically including meaningful decision support, explainability, privacy, consent, algorithmic bias, security, scalability, the role of corporations, and the changing nature of health care work. Results: Ultimately, we suggest that the future of ML in health care remains positive but uncertain, as support from patients, the public, and a wide range of health care stakeholders is necessary to enable its meaningful implementation. Conclusions: If the implementation science community is to facilitate the adoption of ML in ways that stand to generate widespread benefits, the issues raised in this paper will require substantial attention in the coming years. %M 31293245 %R 10.2196/13659 %U https://www.jmir.org/2019/7/e13659/ %U https://doi.org/10.2196/13659 %U http://www.ncbi.nlm.nih.gov/pubmed/31293245 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 7 %P e13664 %T Reducing Patient Loneliness With Artificial Agents: Design Insights From Evolutionary Neuropsychiatry %A Loveys,Kate %A Fricchione,Gregory %A Kolappa,Kavitha %A Sagar,Mark %A Broadbent,Elizabeth %+ Department of Psychological Medicine, The University of Auckland, Auckland City Hospital, Level 12 Support Building, 85 Park Road, Grafton, Auckland, 1023, New Zealand, 64 9 373 7599 ext 84340, k.loveys@auckland.ac.nz %K loneliness %K neuropsychiatry %K biological evolution %K psychological bonding %K interpersonal relations %K artificial intelligence %K social support %K eHealth %D 2019 %7 08.07.2019 %9 Viewpoint %J J Med Internet Res %G English %X Loneliness is a growing public health issue that substantially increases the risk of morbidity and mortality. Artificial agents, such as robots, embodied conversational agents, and chatbots, present an innovation in care delivery and have been shown to reduce patient loneliness by providing social support. However, similar to doctor and patient relationships, the quality of a patient’s relationship with an artificial agent can impact support effectiveness as well as care engagement. Incorporating mammalian attachment-building behavior in neural network processing as part of an agent’s capabilities may improve relationship quality and engagement between patients and artificial agents. We encourage developers of artificial agents intended to relieve patient loneliness to incorporate design insights from evolutionary neuropsychiatry. %M 31287067 %R 10.2196/13664 %U https://www.jmir.org/2019/7/e13664/ %U https://doi.org/10.2196/13664 %U http://www.ncbi.nlm.nih.gov/pubmed/31287067 %0 Journal Article %@ 2369-3762 %I JMIR Publications %V 5 %N 1 %P e13930 %T Applications and Challenges of Implementing Artificial Intelligence in Medical Education: Integrative Review %A Chan,Kai Siang %A Zary,Nabil %+ Mohammed Bin Rashid University of Medicine and Health Sciences, Building 14, Dubai Healthcare City, PO Box 505055, Dubai,, United Arab Emirates, 971 83571846, nabil.zary@icloud.com %K medical education %K evaluation of AIED systems %K real world applications of AIED systems %K artificial intelligence %D 2019 %7 15.6.2019 %9 Review %J JMIR Med Educ %G English %X Background: Since the advent of artificial intelligence (AI) in 1955, the applications of AI have increased over the years within a rapidly changing digital landscape where public expectations are on the rise, fed by social media, industry leaders, and medical practitioners. However, there has been little interest in AI in medical education until the last two decades, with only a recent increase in the number of publications and citations in the field. To our knowledge, thus far, a limited number of articles have discussed or reviewed the current use of AI in medical education. Objective: This study aims to review the current applications of AI in medical education as well as the challenges of implementing AI in medical education. Methods: Medline (Ovid), EBSCOhost Education Resources Information Center (ERIC) and Education Source, and Web of Science were searched with explicit inclusion and exclusion criteria. Full text of the selected articles was analyzed using the Extension of Technology Acceptance Model and the Diffusions of Innovations theory. Data were subsequently pooled together and analyzed quantitatively. Results: A total of 37 articles were identified. Three primary uses of AI in medical education were identified: learning support (n=32), assessment of students’ learning (n=4), and curriculum review (n=1). The main reasons for use of AI are its ability to provide feedback and a guided learning pathway and to decrease costs. Subgroup analysis revealed that medical undergraduates are the primary target audience for AI use. In addition, 34 articles described the challenges of AI implementation in medical education; two main reasons were identified: difficulty in assessing the effectiveness of AI in medical education and technical challenges while developing AI applications. Conclusions: The primary use of AI in medical education was for learning support mainly due to its ability to provide individualized feedback. Little emphasis was placed on curriculum review and assessment of students’ learning due to the lack of digitalization and sensitive nature of examinations, respectively. Big data manipulation also warrants the need to ensure data integrity. Methodological improvements are required to increase AI adoption by addressing the technical difficulties of creating an AI application and using novel methods to assess the effectiveness of AI. To better integrate AI into the medical profession, measures should be taken to introduce AI into the medical school curriculum for medical professionals to better understand AI algorithms and maximize its use. %M 31199295 %R 10.2196/13930 %U http://mededu.jmir.org/2019/1/e13930/ %U https://doi.org/10.2196/13930 %U http://www.ncbi.nlm.nih.gov/pubmed/31199295 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 5 %P e13216 %T Your Robot Therapist Will See You Now: Ethical Implications of Embodied Artificial Intelligence in Psychiatry, Psychology, and Psychotherapy %A Fiske,Amelia %A Henningsen,Peter %A Buyx,Alena %+ Institute for History and Ethics of Medicine, Technical University of Munich School of Medicine, Technical University of Munich, Ismaninger Straße 22, Munich, 81675, Germany, 49 8941404041, a.fiske@tum.de %K artificial intelligence %K robotics %K ethics %K psychiatry %K psychology %K psychotherapy %K medicine %D 2019 %7 09.05.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Research in embodied artificial intelligence (AI) has increasing clinical relevance for therapeutic applications in mental health services. With innovations ranging from ‘virtual psychotherapists’ to social robots in dementia care and autism disorder, to robots for sexual disorders, artificially intelligent virtual and robotic agents are increasingly taking on high-level therapeutic interventions that used to be offered exclusively by highly trained, skilled health professionals. In order to enable responsible clinical implementation, ethical and social implications of the increasing use of embodied AI in mental health need to be identified and addressed. Objective: This paper assesses the ethical and social implications of translating embodied AI applications into mental health care across the fields of Psychiatry, Psychology and Psychotherapy. Building on this analysis, it develops a set of preliminary recommendations on how to address ethical and social challenges in current and future applications of embodied AI. Methods: Based on a thematic literature search and established principles of medical ethics, an analysis of the ethical and social aspects of currently embodied AI applications was conducted across the fields of Psychiatry, Psychology, and Psychotherapy. To enable a comprehensive evaluation, the analysis was structured around the following three steps: assessment of potential benefits; analysis of overarching ethical issues and concerns; discussion of specific ethical and social issues of the interventions. Results: From an ethical perspective, important benefits of embodied AI applications in mental health include new modes of treatment, opportunities to engage hard-to-reach populations, better patient response, and freeing up time for physicians. Overarching ethical issues and concerns include: harm prevention and various questions of data ethics; a lack of guidance on development of AI applications, their clinical integration and training of health professionals; ‘gaps’ in ethical and regulatory frameworks; the potential for misuse including using the technologies to replace established services, thereby potentially exacerbating existing health inequalities. Specific challenges identified and discussed in the application of embodied AI include: matters of risk-assessment, referrals, and supervision; the need to respect and protect patient autonomy; the role of non-human therapy; transparency in the use of algorithms; and specific concerns regarding long-term effects of these applications on understandings of illness and the human condition. Conclusions: We argue that embodied AI is a promising approach across the field of mental health; however, further research is needed to address the broader ethical and societal concerns of these technologies to negotiate best research and medical practices in innovative mental health care. We conclude by indicating areas of future research and developing recommendations for high-priority areas in need of concrete ethical guidance. %M 31094356 %R 10.2196/13216 %U https://www.jmir.org/2019/5/e13216/ %U https://doi.org/10.2196/13216 %U http://www.ncbi.nlm.nih.gov/pubmed/31094356 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 5 %P e11030 %T Data-Driven Blood Glucose Pattern Classification and Anomalies Detection: Machine-Learning Applications in Type 1 Diabetes %A Woldaregay,Ashenafi Zebene %A Årsand,Eirik %A Botsis,Taxiarchis %A Albers,David %A Mamykina,Lena %A Hartvigsen,Gunnar %+ Department of Computer Science, University of Tromsø – The Arctic University of Norway, Realfagbygget, Hansine Hansens vei 54, Tromsø,, Norway, 47 77646444, ashenafi.z.woldaregay@uit.no %K type 1 diabetes %K blood glucose dynamics %K anomalies detection %K machine learning %D 2019 %7 01.05.2019 %9 Review %J J Med Internet Res %G English %X Background: Diabetes mellitus is a chronic metabolic disorder that results in abnormal blood glucose (BG) regulations. The BG level is preferably maintained close to normality through self-management practices, which involves actively tracking BG levels and taking proper actions including adjusting diet and insulin medications. BG anomalies could be defined as any undesirable reading because of either a precisely known reason (normal cause variation) or an unknown reason (special cause variation) to the patient. Recently, machine-learning applications have been widely introduced within diabetes research in general and BG anomaly detection in particular. However, irrespective of their expanding and increasing popularity, there is a lack of up-to-date reviews that materialize the current trends in modeling options and strategies for BG anomaly classification and detection in people with diabetes. Objective: This review aimed to identify, assess, and analyze the state-of-the-art machine-learning strategies and their hybrid systems focusing on BG anomaly classification and detection including glycemic variability (GV), hyperglycemia, and hypoglycemia in type 1 diabetes within the context of personalized decision support systems and BG alarm events applications, which are important constituents for optimal diabetes self-management. Methods: A rigorous literature search was conducted between September 1 and October 1, 2017, and October 15 and November 5, 2018, through various Web-based databases. Peer-reviewed journals and articles were considered. Information from the selected literature was extracted based on predefined categories, which were based on previous research and further elaborated through brainstorming. Results: The initial results were vetted using the title, abstract, and keywords and retrieved 496 papers. After a thorough assessment and screening, 47 articles remained, which were critically analyzed. The interrater agreement was measured using a Cohen kappa test, and disagreements were resolved through discussion. The state-of-the-art classes of machine learning have been developed and tested up to the task and achieved promising performance including artificial neural network, support vector machine, decision tree, genetic algorithm, Gaussian process regression, Bayesian neural network, deep belief network, and others. Conclusions: Despite the complexity of BG dynamics, there are many attempts to capture hypoglycemia and hyperglycemia incidences and the extent of an individual’s GV using different approaches. Recently, the advancement of diabetes technologies and continuous accumulation of self-collected health data have paved the way for popularity of machine learning in these tasks. According to the review, most of the identified studies used a theoretical threshold, which suffers from inter- and intrapatient variation. Therefore, future studies should consider the difference among patients and also track its temporal change over time. Moreover, studies should also give more emphasis on the types of inputs used and their associated time lag. Generally, we foresee that these developments might encourage researchers to further develop and test these systems on a large-scale basis. %M 31042157 %R 10.2196/11030 %U https://www.jmir.org/2019/5/e11030/ %U https://doi.org/10.2196/11030 %U http://www.ncbi.nlm.nih.gov/pubmed/31042157 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 2 %P e13445 %T The Use of Artificially Intelligent Self-Diagnosing Digital Platforms by the General Public: Scoping Review %A Aboueid,Stephanie %A Liu,Rebecca H %A Desta,Binyam Negussie %A Chaurasia,Ashok %A Ebrahim,Shanil %+ Applied Health Sciences, University of Waterloo, 200 University Avenue West, Waterloo, ON, N2L 3G5, Canada, 1 6134061899, seaboueid@uwaterloo.ca %K diagnosis %K artificial intelligence %K symptom checkers %K diagnostic self evaluation %K self-care %D 2019 %7 01.05.2019 %9 Review %J JMIR Med Inform %G English %X Background: Self-diagnosis is the process of diagnosing or identifying a medical condition in oneself. Artificially intelligent digital platforms for self-diagnosis are becoming widely available and are used by the general public; however, little is known about the body of knowledge surrounding this technology. Objective: The objectives of this scoping review were to (1) systematically map the extent and nature of the literature and topic areas pertaining to digital platforms that use computerized algorithms to provide users with a list of potential diagnoses and (2) identify key knowledge gaps. Methods: The following databases were searched: PubMed (Medline), Scopus, Association for Computing Machinery Digital Library, Institute of Electrical and Electronics Engineers, Google Scholar, Open Grey, and ProQuest Dissertations and Theses. The search strategy was developed and refined with the assistance of a librarian and consisted of 3 main concepts: (1) self-diagnosis; (2) digital platforms; and (3) public or patients. The search generated 2536 articles from which 217 were duplicates. Following the Tricco et al 2018 checklist, 2 researchers screened the titles and abstracts (n=2316) and full texts (n=104), independently. A total of 19 articles were included for review, and data were retrieved following a data-charting form that was pretested by the research team. Results: The included articles were mainly conducted in the United States (n=10) or the United Kingdom (n=4). Among the articles, topic areas included accuracy or correspondence with a doctor’s diagnosis (n=6), commentaries (n=2), regulation (n=3), sociological (n=2), user experience (n=2), theoretical (n=1), privacy and security (n=1), ethical (n=1), and design (n=1). Individuals who do not have access to health care and perceive to have a stigmatizing condition are more likely to use this technology. The accuracy of this technology varied substantially based on the disease examined and platform used. Women and those with higher education were more likely to choose the right diagnosis out of the potential list of diagnoses. Regulation of this technology is lacking in most parts of the world; however, they are currently under development. Conclusions: There are prominent research gaps in the literature surrounding the use of artificially intelligent self-diagnosing digital platforms. Given the variety of digital platforms and the wide array of diseases they cover, measuring accuracy is cumbersome. More research is needed to understand the user experience and inform regulations. %M 31042151 %R 10.2196/13445 %U http://medinform.jmir.org/2019/2/e13445/ %U https://doi.org/10.2196/13445 %U http://www.ncbi.nlm.nih.gov/pubmed/31042151 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 4 %P e13822 %T Detecting Developmental Delay and Autism Through Machine Learning Models Using Home Videos of Bangladeshi Children: Development and Validation Study %A Tariq,Qandeel %A Fleming,Scott Lanyon %A Schwartz,Jessey Nicole %A Dunlap,Kaitlyn %A Corbin,Conor %A Washington,Peter %A Kalantarian,Haik %A Khan,Naila Z %A Darmstadt,Gary L %A Wall,Dennis Paul %+ Division of Systems Medicine, Department of Pediatrics, Stanford University, 1265 Welch Road, Palo Alto, CA, 94305, United States, 1 6173946031, dpwall@stanford.edu %K autism %K autism spectrum disorder %K machine learning %K developmental delays %K clinical resources %K Bangladesh %K Biomedical Data Science %D 2019 %7 24.04.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Autism spectrum disorder (ASD) is currently diagnosed using qualitative methods that measure between 20-100 behaviors, can span multiple appointments with trained clinicians, and take several hours to complete. In our previous work, we demonstrated the efficacy of machine learning classifiers to accelerate the process by collecting home videos of US-based children, identifying a reduced subset of behavioral features that are scored by untrained raters using a machine learning classifier to determine children’s “risk scores” for autism. We achieved an accuracy of 92% (95% CI 88%-97%) on US videos using a classifier built on five features. Objective: Using videos of Bangladeshi children collected from Dhaka Shishu Children’s Hospital, we aim to scale our pipeline to another culture and other developmental delays, including speech and language conditions. Methods: Although our previously published and validated pipeline and set of classifiers perform reasonably well on Bangladeshi videos (75% accuracy, 95% CI 71%-78%), this work improves on that accuracy through the development and application of a powerful new technique for adaptive aggregation of crowdsourced labels. We enhance both the utility and performance of our model by building two classification layers: The first layer distinguishes between typical and atypical behavior, and the second layer distinguishes between ASD and non-ASD. In each of the layers, we use a unique rater weighting scheme to aggregate classification scores from different raters based on their expertise. We also determine Shapley values for the most important features in the classifier to understand how the classifiers’ process aligns with clinical intuition. Results: Using these techniques, we achieved an accuracy (area under the curve [AUC]) of 76% (SD 3%) and sensitivity of 76% (SD 4%) for identifying atypical children from among developmentally delayed children, and an accuracy (AUC) of 85% (SD 5%) and sensitivity of 76% (SD 6%) for identifying children with ASD from those predicted to have other developmental delays. Conclusions: These results show promise for using a mobile video-based and machine learning–directed approach for early and remote detection of autism in Bangladeshi children. This strategy could provide important resources for developmental health in developing countries with few clinical resources for diagnosis, helping children get access to care at an early age. Future research aimed at extending the application of this approach to identify a range of other conditions and determine the population-level burden of developmental disabilities and impairments will be of high value. %M 31017583 %R 10.2196/13822 %U http://www.jmir.org/2019/4/e13822/ %U https://doi.org/10.2196/13822 %U http://www.ncbi.nlm.nih.gov/pubmed/31017583 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 4 %P e12887 %T Physicians’ Perceptions of Chatbots in Health Care: Cross-Sectional Web-Based Survey %A Palanica,Adam %A Flaschner,Peter %A Thommandram,Anirudh %A Li,Michael %A Fossat,Yan %+ Labs Department, Klick Health, Klick Inc, 175 Bloor St E, Suite 300, Toronto, ON, M4W 3R8, Canada, 1 416 214 4977, apalanica@klick.com %K physician satisfaction %K health care %K telemedicine %K mobile health %K health surveys %D 2019 %7 05.04.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Many potential benefits for the uses of chatbots within the context of health care have been theorized, such as improved patient education and treatment compliance. However, little is known about the perspectives of practicing medical physicians on the use of chatbots in health care, even though these individuals are the traditional benchmark of proper patient care. Objective: This study aimed to investigate the perceptions of physicians regarding the use of health care chatbots, including their benefits, challenges, and risks to patients. Methods: A total of 100 practicing physicians across the United States completed a Web-based, self-report survey to examine their opinions of chatbot technology in health care. Descriptive statistics and frequencies were used to examine the characteristics of participants. Results: A wide variety of positive and negative perspectives were reported on the use of health care chatbots, including the importance to patients for managing their own health and the benefits on physical, psychological, and behavioral health outcomes. More consistent agreement occurred with regard to administrative benefits associated with chatbots; many physicians believed that chatbots would be most beneficial for scheduling doctor appointments (78%, 78/100), locating health clinics (76%, 76/100), or providing medication information (71%, 71/100). Conversely, many physicians believed that chatbots cannot effectively care for all of the patients’ needs (76%, 76/100), cannot display human emotion (72%, 72/100), and cannot provide detailed diagnosis and treatment because of not knowing all of the personal factors associated with the patient (71%, 71/100). Many physicians also stated that health care chatbots could be a risk to patients if they self-diagnose too often (714%, 74/100) and do not accurately understand the diagnoses (74%, 74/100). Conclusions: Physicians believed in both costs and benefits associated with chatbots, depending on the logistics and specific roles of the technology. Chatbots may have a beneficial role to play in health care to support, motivate, and coach patients as well as for streamlining organizational tasks; in essence, chatbots could become a surrogate for nonmedical caregivers. However, concerns remain on the inability of chatbots to comprehend the emotional state of humans as well as in areas where expert medical knowledge and intelligence is required. %M 30950796 %R 10.2196/12887 %U https://www.jmir.org/2019/4/e12887/ %U https://doi.org/10.2196/12887 %U http://www.ncbi.nlm.nih.gov/pubmed/30950796 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 4 %P e12286 %T Applications of Machine Learning in Real-Life Digital Health Interventions: Review of the Literature %A Triantafyllidis,Andreas K %A Tsanas,Athanasios %+ Information Technologies Institute, Centre for Research and Technology Hellas, 6th km Charilaou-Thermi Rd, GR 57001 Thermi, Thessaloniki, 60361, Greece, 30 2310 498100, atriand@gmail.com %K machine learning %K data mining %K artificial intelligence %K digital health %K review %K telemedicine %D 2019 %7 05.04.2019 %9 Review %J J Med Internet Res %G English %X Background: Machine learning has attracted considerable research interest toward developing smart digital health interventions. These interventions have the potential to revolutionize health care and lead to substantial outcomes for patients and medical professionals. Objective: Our objective was to review the literature on applications of machine learning in real-life digital health interventions, aiming to improve the understanding of researchers, clinicians, engineers, and policy makers in developing robust and impactful data-driven interventions in the health care domain. Methods: We searched the PubMed and Scopus bibliographic databases with terms related to machine learning, to identify real-life studies of digital health interventions incorporating machine learning algorithms. We grouped those interventions according to their target (ie, target condition), study design, number of enrolled participants, follow-up duration, primary outcome and whether this had been statistically significant, machine learning algorithms used in the intervention, and outcome of the algorithms (eg, prediction). Results: Our literature search identified 8 interventions incorporating machine learning in a real-life research setting, of which 3 (37%) were evaluated in a randomized controlled trial and 5 (63%) in a pilot or experimental single-group study. The interventions targeted depression prediction and management, speech recognition for people with speech disabilities, self-efficacy for weight loss, detection of changes in biopsychosocial condition of patients with multiple morbidity, stress management, treatment of phantom limb pain, smoking cessation, and personalized nutrition based on glycemic response. The average number of enrolled participants in the studies was 71 (range 8-214), and the average follow-up study duration was 69 days (range 3-180). Of the 8 interventions, 6 (75%) showed statistical significance (at the P=.05 level) in health outcomes. Conclusions: This review found that digital health interventions incorporating machine learning algorithms in real-life studies can be useful and effective. Given the low number of studies identified in this review and that they did not follow a rigorous machine learning evaluation methodology, we urge the research community to conduct further studies in intervention settings following evaluation principles and demonstrating the potential of machine learning in clinical practice. %M 30950797 %R 10.2196/12286 %U https://www.jmir.org/2019/4/e12286/ %U https://doi.org/10.2196/12286 %U http://www.ncbi.nlm.nih.gov/pubmed/30950797 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 8 %N 2 %P e12100 %T Artificial Intelligence in Clinical Health Care Applications: Viewpoint %A van Hartskamp,Michael %A Consoli,Sergio %A Verhaegh,Wim %A Petkovic,Milan %A van de Stolpe,Anja %+ Philips Research, HTC11, p247, High Tech Campus, Eindhoven, 5656AE, Netherlands, 31 612784841, anja.van.de.stolpe@philips.com %K artificial intelligence %K deep learning %K clinical data %K Bayesian modeling %K medical informatics %D 2019 %7 05.04.2019 %9 Viewpoint %J Interact J Med Res %G English %X The idea of artificial intelligence (AI) has a long history. It turned out, however, that reaching intelligence at human levels is more complicated than originally anticipated. Currently, we are experiencing a renewed interest in AI, fueled by an enormous increase in computing power and an even larger increase in data, in combination with improved AI technologies like deep learning. Healthcare is considered the next domain to be revolutionized by artificial intelligence. While AI approaches are excellently suited to develop certain algorithms, for biomedical applications there are specific challenges. We propose six recommendations—the 6Rs—to improve AI projects in the biomedical space, especially clinical health care, and to facilitate communication between AI scientists and medical doctors: (1) Relevant and well-defined clinical question first; (2) Right data (ie, representative and of good quality); (3) Ratio between number of patients and their variables should fit the AI method; (4) Relationship between data and ground truth should be as direct and causal as possible; (5) Regulatory ready; enabling validation; and (6) Right AI method. %M 30950806 %R 10.2196/12100 %U https://www.i-jmr.org/2019/2/e12100/ %U https://doi.org/10.2196/12100 %U http://www.ncbi.nlm.nih.gov/pubmed/30950806 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 3 %P e12422 %T Physician Confidence in Artificial Intelligence: An Online Mobile Survey %A Oh,Songhee %A Kim,Jae Heon %A Choi,Sung-Woo %A Lee,Hee Jeong %A Hong,Jungrak %A Kwon,Soon Hyo %+ Division of Nephrology, Department of Internal Medicine, Soonchunhyang University Hospital, Daesagwanro 59 Youngsangu, Seoul, 04401, Republic of Korea, 82 01034413147, ksoonhyo@schmc.ac.kr %K artificial intelligence %K AI %K awareness %K physicians %D 2019 %7 25.03.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: It is expected that artificial intelligence (AI) will be used extensively in the medical field in the future. Objective: The purpose of this study is to investigate the awareness of AI among Korean doctors and to assess physicians’ attitudes toward the medical application of AI. Methods: We conducted an online survey composed of 11 closed-ended questions using Google Forms. The survey consisted of questions regarding the recognition of and attitudes toward AI, the development direction of AI in medicine, and the possible risks of using AI in the medical field. Results: A total of 669 participants completed the survey. Only 40 (5.9%) answered that they had good familiarity with AI. However, most participants considered AI useful in the medical field (558/669, 83.4% agreement). The advantage of using AI was seen as the ability to analyze vast amounts of high-quality, clinically relevant data in real time. Respondents agreed that the area of medicine in which AI would be most useful is disease diagnosis (558/669, 83.4% agreement). One possible problem cited by the participants was that AI would not be able to assist in unexpected situations owing to inadequate information (196/669, 29.3%). Less than half of the participants(294/669, 43.9%) agreed that AI is diagnostically superior to human doctors. Only 237 (35.4%) answered that they agreed that AI could replace them in their jobs. Conclusions: This study suggests that Korean doctors and medical students have favorable attitudes toward AI in the medical field. The majority of physicians surveyed believed that AI will not replace their roles in the future. %M 30907742 %R 10.2196/12422 %U http://www.jmir.org/2019/3/e12422/ %U https://doi.org/10.2196/12422 %U http://www.ncbi.nlm.nih.gov/pubmed/30907742 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 3 %P e12802 %T Artificial Intelligence and the Future of Primary Care: Exploratory Qualitative Study of UK General Practitioners’ Views %A Blease,Charlotte %A Kaptchuk,Ted J %A Bernstein,Michael H %A Mandl,Kenneth D %A Halamka,John D %A DesRoches,Catherine M %+ General Medicine and Primary Care, Beth Israel Deaconess Medical Center, Harvard Medical School, 330 Brookline Avenue, Boston, MA, MA 02215, United States, 1 617 754 1457, cblease@bidmc.harvard.edu %K artificial intelligence %K attitudes %K future %K general practice %K machine learning %K opinions %K primary care %K qualitative research %K technology %D 2019 %7 20.03.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: The potential for machine learning to disrupt the medical profession is the subject of ongoing debate within biomedical informatics and related fields. Objective: This study aimed to explore general practitioners’ (GPs’) opinions about the potential impact of future technology on key tasks in primary care. Methods: In June 2018, we conducted a Web-based survey of 720 UK GPs’ opinions about the likelihood of future technology to fully replace GPs in performing 6 key primary care tasks, and, if respondents considered replacement for a particular task likely, to estimate how soon the technological capacity might emerge. This study involved qualitative descriptive analysis of written responses (“comments”) to an open-ended question in the survey. Results: Comments were classified into 3 major categories in relation to primary care: (1) limitations of future technology, (2) potential benefits of future technology, and (3) social and ethical concerns. Perceived limitations included the beliefs that communication and empathy are exclusively human competencies; many GPs also considered clinical reasoning and the ability to provide value-based care as necessitating physicians’ judgments. Perceived benefits of technology included expectations about improved efficiencies, in particular with respect to the reduction of administrative burdens on physicians. Social and ethical concerns encompassed multiple, divergent themes including the need to train more doctors to overcome workforce shortfalls and misgivings about the acceptability of future technology to patients. However, some GPs believed that the failure to adopt technological innovations could incur harms to both patients and physicians. Conclusions: This study presents timely information on physicians’ views about the scope of artificial intelligence (AI) in primary care. Overwhelmingly, GPs considered the potential of AI to be limited. These views differ from the predictions of biomedical informaticians. More extensive, stand-alone qualitative work would provide a more in-depth understanding of GPs’ views. %M 30892270 %R 10.2196/12802 %U http://www.jmir.org/2019/3/e12802/ %U https://doi.org/10.2196/12802 %U http://www.ncbi.nlm.nih.gov/pubmed/30892270 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 3 %P e11990 %T Detecting Hypoglycemia Incidents Reported in Patients’ Secure Messages: Using Cost-Sensitive Learning and Oversampling to Reduce Data Imbalance %A Chen,Jinying %A Lalor,John %A Liu,Weisong %A Druhl,Emily %A Granillo,Edgard %A Vimalananda,Varsha G %A Yu,Hong %+ Department of Population and Quantitative Health Sciences, University of Massachusetts Medical School, Albert Sherman Center, 9th Floor, 368 Plantation Street, Worcester, MA, 01605, United States, 1 508 856 6063, jinying.chen@umassmed.edu %K secure messaging %K natural language processing %K hypoglycemia %K supervised machine learning %K imbalanced data %K adverse event detection %K drug-related side effects and adverse reactions %D 2019 %7 11.03.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Improper dosing of medications such as insulin can cause hypoglycemic episodes, which may lead to severe morbidity or even death. Although secure messaging was designed for exchanging nonurgent messages, patients sometimes report hypoglycemia events through secure messaging. Detecting these patient-reported adverse events may help alert clinical teams and enable early corrective actions to improve patient safety. Objective: We aimed to develop a natural language processing system, called HypoDetect (Hypoglycemia Detector), to automatically identify hypoglycemia incidents reported in patients’ secure messages. Methods: An expert in public health annotated 3000 secure message threads between patients with diabetes and US Department of Veterans Affairs clinical teams as containing patient-reported hypoglycemia incidents or not. A physician independently annotated 100 threads randomly selected from this dataset to determine interannotator agreement. We used this dataset to develop and evaluate HypoDetect. HypoDetect incorporates 3 machine learning algorithms widely used for text classification: linear support vector machines, random forest, and logistic regression. We explored different learning features, including new knowledge-driven features. Because only 114 (3.80%) messages were annotated as positive, we investigated cost-sensitive learning and oversampling methods to mitigate the challenge of imbalanced data. Results: The interannotator agreement was Cohen kappa=.976. Using cross-validation, logistic regression with cost-sensitive learning achieved the best performance (area under the receiver operating characteristic curve=0.954, sensitivity=0.693, specificity 0.974, F1 score=0.590). Cost-sensitive learning and the ensembled synthetic minority oversampling technique improved the sensitivity of the baseline systems substantially (by 0.123 to 0.728 absolute gains). Our results show that a variety of features contributed to the best performance of HypoDetect. Conclusions: Despite the challenge of data imbalance, HypoDetect achieved promising results for the task of detecting hypoglycemia incidents from secure messages. The system has a great potential to facilitate early detection and treatment of hypoglycemia. %M 30855231 %R 10.2196/11990 %U http://www.jmir.org/2019/3/e11990/ %U https://doi.org/10.2196/11990 %U http://www.ncbi.nlm.nih.gov/pubmed/30855231 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 1 %P e10788 %T Detection of Bleeding Events in Electronic Health Record Notes Using Convolutional Neural Network Models Enhanced With Recurrent Neural Network Autoencoders: Deep Learning Approach %A Li,Rumeng %A Hu,Baotian %A Liu,Feifan %A Liu,Weisong %A Cunningham,Francesca %A McManus,David D %A Yu,Hong %+ Department of Computer Science, University of Massachusetts Lowell, 1 University Avenue, Lowell, MA, 01854, United States, 1 9789343620, hong_yu@uml.edu %K autoencoder %K BiLSTM %K bleeding %K convolutional neural networks %K electronic health record %D 2019 %7 08.02.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: Bleeding events are common and critical and may cause significant morbidity and mortality. High incidences of bleeding events are associated with cardiovascular disease in patients on anticoagulant therapy. Prompt and accurate detection of bleeding events is essential to prevent serious consequences. As bleeding events are often described in clinical notes, automatic detection of bleeding events from electronic health record (EHR) notes may improve drug-safety surveillance and pharmacovigilance. Objective: We aimed to develop a natural language processing (NLP) system to automatically classify whether an EHR note sentence contains a bleeding event. Methods: We expert annotated 878 EHR notes (76,577 sentences and 562,630 word-tokens) to identify bleeding events at the sentence level. This annotated corpus was used to train and validate our NLP systems. We developed an innovative hybrid convolutional neural network (CNN) and long short-term memory (LSTM) autoencoder (HCLA) model that integrates a CNN architecture with a bidirectional LSTM (BiLSTM) autoencoder model to leverage large unlabeled EHR data. Results: HCLA achieved the best area under the receiver operating characteristic curve (0.957) and F1 score (0.938) to identify whether a sentence contains a bleeding event, thereby surpassing the strong baseline support vector machines and other CNN and autoencoder models. Conclusions: By incorporating a supervised CNN model and a pretrained unsupervised BiLSTM autoencoder, the HCLA achieved high performance in detecting bleeding events. %M 30735140 %R 10.2196/10788 %U http://medinform.jmir.org/2019/1/e10788/ %U https://doi.org/10.2196/10788 %U http://www.ncbi.nlm.nih.gov/pubmed/30735140 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 5 %N 4 %P e64 %T Using Psychological Artificial Intelligence (Tess) to Relieve Symptoms of Depression and Anxiety: Randomized Controlled Trial %A Fulmer,Russell %A Joerin,Angela %A Gentile,Breanna %A Lakerink,Lysanne %A Rauws,Michiel %+ Northwestern University, 633 Clark Street, Evanston, IL, United States, 1 312 609 5300 ext 699, russell.fulmer@northwestern.edu %K artificial intelligence %K mental health services %K depression %K anxiety %K students %D 2018 %7 13.12.2018 %9 Original Paper %J JMIR Ment Health %G English %X Background: Students in need of mental health care face many barriers including cost, location, availability, and stigma. Studies show that computer-assisted therapy and 1 conversational chatbot delivering cognitive behavioral therapy (CBT) offer a less-intensive and more cost-effective alternative for treating depression and anxiety. Although CBT is one of the most effective treatment methods, applying an integrative approach has been linked to equally effective posttreatment improvement. Integrative psychological artificial intelligence (AI) offers a scalable solution as the demand for affordable, convenient, lasting, and secure support grows. Objective: This study aimed to assess the feasibility and efficacy of using an integrative psychological AI, Tess, to reduce self-identified symptoms of depression and anxiety in college students. Methods: In this randomized controlled trial, 75 participants were recruited from 15 universities across the United States. All participants completed Web-based surveys, including the Patient Health Questionnaire (PHQ-9), Generalized Anxiety Disorder Scale (GAD-7), and Positive and Negative Affect Scale (PANAS) at baseline and 2 to 4 weeks later (T2). The 2 test groups consisted of 50 participants in total and were randomized to receive unlimited access to Tess for either 2 weeks (n=24) or 4 weeks (n=26). The information-only control group participants (n=24) received an electronic link to the National Institute of Mental Health’s (NIMH) eBook on depression among college students and were only granted access to Tess after completion of the study. Results: A sample of 74 participants completed this study with 0% attrition from the test group and less than 1% attrition from the control group (1/24). The average age of participants was 22.9 years, with 70% of participants being female (52/74), mostly Asian (37/74, 51%), and white (32/74, 41%). Group 1 received unlimited access to Tess, with daily check-ins for 2 weeks. Group 2 received unlimited access to Tess with biweekly check-ins for 4 weeks. The information-only control group was provided with an electronic link to the NIMH’s eBook. Multivariate analysis of covariance was conducted. We used an alpha level of .05 for all statistical tests. Results revealed a statistically significant difference between the control group and group 1, such that group 1 reported a significant reduction in symptoms of depression as measured by the PHQ-9 (P=.03), whereas those in the control group did not. A statistically significant difference was found between the control group and both test groups 1 and 2 for symptoms of anxiety as measured by the GAD-7. Group 1 (P=.045) and group 2 (P=.02) reported a significant reduction in symptoms of anxiety, whereas the control group did not. A statistically significant difference was found on the PANAS between the control group and group 1 (P=.03) and suggests that Tess did impact scores. Conclusions: This study offers evidence that AI can serve as a cost-effective and accessible therapeutic agent. Although not designed to appropriate the role of a trained therapist, integrative psychological AI emerges as a feasible option for delivering support. Trial Registration: International Standard Randomized Controlled Trial Number: ISRCTN61214172; https://doi.org/10.1186/ISRCTN61214172. %M 30545815 %R 10.2196/mental.9782 %U http://mental.jmir.org/2018/4/e64/ %U https://doi.org/10.2196/mental.9782 %U http://www.ncbi.nlm.nih.gov/pubmed/30545815 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 6 %N 11 %P e198 %T The Perceived Benefits of an Artificial Intelligence–Embedded Mobile App Implementing Evidence-Based Guidelines for the Self-Management of Chronic Neck and Back Pain: Observational Study %A Lo,Wai Leung Ambrose %A Lei,Di %A Li,Le %A Huang,Dong Feng %A Tong,Kin-Fai %+ Guangdong Engineering and Technology Research Center for Rehabilitation Medicine and Translation, Department of Rehabilitation Medicine, The First Affiliated Hospital, Sun Yat-sen University, 58 Zhongshan Second Road, Guangzhou, 510000, China, 86 87332200 ext 8536, ambroselo0726@outlook.com %K low back pain %K neck pain %K mobile app %K exercise therapy %K mHealth %D 2018 %7 26.11.2018 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: Chronic musculoskeletal neck and back pain are disabling conditions among adults. Use of technology has been suggested as an alternative way to increase adherence to exercise therapy, which may improve clinical outcomes. Objective: The aim was to investigate the self-perceived benefits of an artificial intelligence (AI)–embedded mobile app to self-manage chronic neck and back pain. Methods: A total of 161 participants responded to the invitation. The evaluation questionnaire included 14 questions that were intended to explore if using the AI rehabilitation system may (1) increase time spent on therapeutic exercise, (2) affect pain level (assessed by the 0-10 Numerical Pain Rating Scale), and (3) reduce the need for other interventions. Results: An increase in time spent on therapeutic exercise per day was observed. The median Numerical Pain Rating Scale scores were 6 (interquartile range [IQR] 5-8) before and 4 (IQR 3-6) after using the AI-embedded mobile app (95% CI 1.18-1.81). A 3-point reduction was reported by the participants who used the AI-embedded mobile app for more than 6 months. Reduction in the usage of other interventions while using the AI-embedded mobile app was also reported. Conclusions: This study demonstrated the positive self-perceived beneficiary effect of using the AI-embedded mobile app to provide a personalized therapeutic exercise program. The positive results suggest that it at least warrants further study to investigate the physiological effect of the AI-embedded mobile app and how it compares with routine clinical care. %M 30478019 %R 10.2196/mhealth.8127 %U http://mhealth.jmir.org/2018/11/e198/ %U https://doi.org/10.2196/mhealth.8127 %U http://www.ncbi.nlm.nih.gov/pubmed/30478019 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 9 %P e11510 %T Patient and Consumer Safety Risks When Using Conversational Assistants for Medical Information: An Observational Study of Siri, Alexa, and Google Assistant %A Bickmore,Timothy W %A Trinh,Ha %A Olafsson,Stefan %A O'Leary,Teresa K %A Asadi,Reza %A Rickles,Nathaniel M %A Cruz,Ricardo %+ College of Computer and Information Science, Northeastern University, 910-177, 360 Huntington Avenue, Boston, MA, 02115, United States, 1 6173735477, bickmore@ccs.neu.edu %K conversational assistant %K conversational interface %K dialogue system %K medical error %K patient safety %D 2018 %7 04.09.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: Conversational assistants, such as Siri, Alexa, and Google Assistant, are ubiquitous and are beginning to be used as portals for medical services. However, the potential safety issues of using conversational assistants for medical information by patients and consumers are not understood. Objective: To determine the prevalence and nature of the harm that could result from patients or consumers using conversational assistants for medical information. Methods: Participants were given medical problems to pose to Siri, Alexa, or Google Assistant, and asked to determine an action to take based on information from the system. Assignment of tasks and systems were randomized across participants, and participants queried the conversational assistants in their own words, making as many attempts as needed until they either reported an action to take or gave up. Participant-reported actions for each medical task were rated for patient harm using an Agency for Healthcare Research and Quality harm scale. Results: Fifty-four subjects completed the study with a mean age of 42 years (SD 18). Twenty-nine (54%) were female, 31 (57%) Caucasian, and 26 (50%) were college educated. Only 8 (15%) reported using a conversational assistant regularly, while 22 (41%) had never used one, and 24 (44%) had tried one “a few times.“ Forty-four (82%) used computers regularly. Subjects were only able to complete 168 (43%) of their 394 tasks. Of these, 49 (29%) reported actions that could have resulted in some degree of patient harm, including 27 (16%) that could have resulted in death. Conclusions: Reliance on conversational assistants for actionable medical information represents a safety risk for patients and consumers. Patients should be cautioned to not use these technologies for answers to medical questions they intend to act on without further consultation from a health care provider. %M 30181110 %R 10.2196/11510 %U http://www.jmir.org/2018/9/e11510/ %U https://doi.org/10.2196/11510 %U http://www.ncbi.nlm.nih.gov/pubmed/30181110 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 5 %N 3 %P e10454 %T An Embodied Conversational Agent for Unguided Internet-Based Cognitive Behavior Therapy in Preventative Mental Health: Feasibility and Acceptability Pilot Trial %A Suganuma,Shinichiro %A Sakamoto,Daisuke %A Shimoyama,Haruhiko %+ Department of Clinical Psychology, Graduate School of Education, The University of Tokyo, , Tokyo,, Japan, 81 3 5841 8068, sgnm.shin@gmail.com %K embodied conversational agent %K cognitive behavioral therapy %K psychological distress %K mental well‐being %K artificial intelligence technology %D 2018 %7 31.07.2018 %9 Original Paper %J JMIR Ment Health %G English %X Background: Recent years have seen an increase in the use of internet-based cognitive behavioral therapy in the area of mental health. Although lower effectiveness and higher dropout rates of unguided than those of guided internet-based cognitive behavioral therapy remain critical issues, not incurring ongoing human clinical resources makes it highly advantageous. Objective: Current research in psychotherapy, which acknowledges the importance of therapeutic alliance, aims to evaluate the feasibility and acceptability, in terms of mental health, of an application that is embodied with a conversational agent. This application was enabled for use as an internet-based cognitive behavioral therapy preventative mental health measure. Methods: Analysis of the data from the 191 participants of the experimental group with a mean age of 38.07 (SD 10.75) years and the 263 participants of the control group with a mean age of 38.05 (SD 13.45) years using a 2-way factorial analysis of variance (group × time) was performed. Results: There was a significant main effect (P=.02) and interaction for time on the variable of positive mental health (P=.02), and for the treatment group, a significant simple main effect was also found (P=.002). In addition, there was a significant main effect (P=.02) and interaction for time on the variable of negative mental health (P=.005), and for the treatment group, a significant simple main effect was also found (P=.001). Conclusions: This research can be seen to represent a certain level of evidence for the mental health application developed herein, indicating empirically that internet-based cognitive behavioral therapy with the embodied conversational agent can be used in mental health care. In the pilot trial, given the issues related to feasibility and acceptability, it is necessary to pursue higher quality evidence while continuing to further improve the application, based on the findings of the current research. %M 30064969 %R 10.2196/10454 %U http://mental.jmir.org/2018/3/e10454/ %U https://doi.org/10.2196/10454 %U http://www.ncbi.nlm.nih.gov/pubmed/30064969 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 6 %P e10148 %T Towards an Artificially Empathic Conversational Agent for Mental Health Applications: System Design and User Perceptions %A Morris,Robert R %A Kouddous,Kareem %A Kshirsagar,Rohan %A Schueller,Stephen M %+ Koko, 155 Rivington St, New York, NY, 10002, United States, 1 617 851 4967, rob@koko.ai %K conversational agents %K mental health %K empathy %K crowdsourcing %K peer support %D 2018 %7 26.06.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: Conversational agents cannot yet express empathy in nuanced ways that account for the unique circumstances of the user. Agents that possess this faculty could be used to enhance digital mental health interventions. Objective: We sought to design a conversational agent that could express empathic support in ways that might approach, or even match, human capabilities. Another aim was to assess how users might appraise such a system. Methods: Our system used a corpus-based approach to simulate expressed empathy. Responses from an existing pool of online peer support data were repurposed by the agent and presented to the user. Information retrieval techniques and word embeddings were used to select historical responses that best matched a user’s concerns. We collected ratings from 37,169 users to evaluate the system. Additionally, we conducted a controlled experiment (N=1284) to test whether the alleged source of a response (human or machine) might change user perceptions. Results: The majority of responses created by the agent (2986/3770, 79.20%) were deemed acceptable by users. However, users significantly preferred the efforts of their peers (P<.001). This effect was maintained in a controlled study (P=.02), even when the only difference in responses was whether they were framed as coming from a human or a machine. Conclusions: Our system illustrates a novel way for machines to construct nuanced and personalized empathic utterances. However, the design had significant limitations and further research is needed to make this approach viable. Our controlled study suggests that even in ideal conditions, nonhuman agents may struggle to express empathy as well as humans. The ethical implications of empathic agents, as well as their potential iatrogenic effects, are also discussed. %M 29945856 %R 10.2196/10148 %U http://www.jmir.org/2018/6/e10148/ %U https://doi.org/10.2196/10148 %U http://www.ncbi.nlm.nih.gov/pubmed/29945856 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 5 %N 2 %P e32 %T Ethical Issues for Direct-to-Consumer Digital Psychotherapy Apps: Addressing Accountability, Data Protection, and Consent %A Martinez-Martin,Nicole %A Kreitmair,Karola %+ Stanford Center for Biomedical Ethics, 1215 Welch Road, Stanford, CA, 94305, United States, 1 650 723 5760, nicolemz@stanford.edu %K ethics %K ethical issues %K mental health %K technology %K telemedicine %K mHealth %K psychotherapy %D 2018 %7 23.04.2018 %9 Viewpoint %J JMIR Ment Health %G English %X This paper focuses on the ethical challenges presented by direct-to-consumer (DTC) digital psychotherapy services that do not involve oversight by a professional mental health provider. DTC digital psychotherapy services can potentially assist in improving access to mental health care for the many people who would otherwise not have the resources or ability to connect with a therapist. However, the lack of adequate regulation in this area exacerbates concerns over how safety, privacy, accountability, and other ethical obligations to protect an individual in therapy are addressed within these services. In the traditional therapeutic relationship, there are ethical obligations that serve to protect the interests of the client and provide warnings. In contrast, in a DTC therapy app, there are no clear lines of accountability or associated ethical obligations to protect the user seeking mental health services. The types of DTC services that present ethical challenges include apps that use a digital platform to connect users to minimally trained nonprofessional counselors, as well as services that provide counseling steered by artificial intelligence and conversational agents. There is a need for adequate oversight of DTC nonprofessional psychotherapy services and additional empirical research to inform policy that will provide protection to the consumer. %M 29685865 %R 10.2196/mental.9423 %U http://mental.jmir.org/2018/2/e32/ %U https://doi.org/10.2196/mental.9423 %U http://www.ncbi.nlm.nih.gov/pubmed/29685865 %0 Journal Article %@ 2369-6893 %I JMIR Publications %V 3 %N 1 %P e37 %T Feasibility of an Automated System Counselor for Survivors of Sexual Assault %A Howe,Esther %A Pedrelli,Paola %A Morris,Robert %A Nyer,Maren %A Mischoulon,David %A Picard,Rosalind %+ Department of Psychiatry, Massachusetts General Hospital, 6th Floor, 1 Bowdoin Square, Boston, MA,, United States, 1 6176437690, ehowe3@mgh.harvard.edu %K CBT %K web chat %D 2017 %7 22.09.2017 %9 Abstract %J iproc %G English %X Background: Sexual assault (SA) is common and costly to individuals and society, and increases risk of mental health disorders. Stigma and cost of care discourage survivors from seeking help. Norms profiling survivors as heterosexual, cisgendered women dissuade LGBTQIA+ individuals and men from accessing care. Because individuals prefer disclosing sensitive information online rather than in-person, online systems—like instant messaging and chatbots—for counseling may bypass concerns about stigma. These systems’ anonymity may increase disclosure and decrease impression management, the process by which individuals attempt to influence others’ perceptions. Their low cost may expand reach of care. There are no known evidence-based chat platforms for SA survivors. Objective: To examine feasibility of a chat platform with peer and automated system (chatbot) counseling interfaces to provide cognitive reappraisals (a cognitive behavioral therapy technique) to survivors. Methods: Participants are English-speaking, US-based survivors, 18+ years old. Participants are told they will be randomized to chat with a peer or automated system counselor 5 times over 2 weeks. In reality, all participants chat with a peer counselor. Chats employ a modified-for-context evidence-based cognitive reappraisal script developed by Koko, a company offering support services for emotional distress via social networks. At baseline, participants indicate counselor type preference and complete a basic demographic form, the Brief Fear of Negative Evaluation Scale, and self-disclosure items from the International Personality Item Pool. After 5 chats, participants complete questions from the Client Satisfaction Questionnaire (CSQ), Self-Reported Attitudes Toward Agent, and the Working Alliance Inventory. Hypotheses: 1) Online chatting and automated systems will be acceptable and feasible means of delivering cognitive reappraisals to survivors. 2) High impression management (IM≥25) and low self-disclosure (SD≤45) will be associated with preference for an automated system. 3) IM and SD will separately moderate the relationship between counselor assignment and participant satisfaction. Results: Ten participants have completed the study. Recruitment is ongoing. We will enroll 50+ participants by 10/2017 and outline findings at the Connected Health Conference. To date, 70% of participants completed all chats within 24 hours of enrollment, and 60% indicated a pre-chat preference for an automated system, suggesting acceptability of the concept. The post-chat CSQ mean total score of 3.98 on a 5-point Likert scale (1=Poor; 5=Excellent) suggests platform acceptability. Of the 50% reporting high IM, 60% indicated preference for an automated system. Of the 30% reporting low SD, 33% reported preference for an automated system. At recruitment completion, ANOVA analyses will elucidate relationships between IM, SD, and counselor assignment. Correlation and linear regression analyses will show any moderating effect of IM and SD on the relationship between counselor assignment and participant satisfaction. Conclusions: Preliminary results suggest acceptability and feasibility of cognitive reappraisals via chat for survivors, and of the automated system counselor concept. Final results will explore relationships between SD, IM, counselor type, and participant satisfaction to inform the development of new platforms for survivors. %R 10.2196/iproc.8585 %U http://www.iproc.org/2017/1/e37/ %U https://doi.org/10.2196/iproc.8585