%0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 3 %P e14830 %T Fine-Tuning Bidirectional Encoder Representations From Transformers (BERT)–Based Models on Large-Scale Electronic Health Record Notes: An Empirical Study %A Li,Fei %A Jin,Yonghao %A Liu,Weisong %A Rawat,Bhanu Pratap Singh %A Cai,Pengshan %A Yu,Hong %+ Department of Computer Science, University of Massachusetts Lowell, 1 University Avenue, Lowell, MA,, United States, 1 978 934 6132, Hong_Yu@uml.edu %K natural language processing %K entity normalization %K deep learning %K electronic health record note %K BERT %D 2019 %7 12.09.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: The bidirectional encoder representations from transformers (BERT) model has achieved great success in many natural language processing (NLP) tasks, such as named entity recognition and question answering. However, little prior work has explored this model to be used for an important task in the biomedical and clinical domains, namely entity normalization. Objective: We aim to investigate the effectiveness of BERT-based models for biomedical or clinical entity normalization. In addition, our second objective is to investigate whether the domains of training data influence the performances of BERT-based models as well as the degree of influence. Methods: Our data was comprised of 1.5 million unlabeled electronic health record (EHR) notes. We first fine-tuned BioBERT on this large collection of unlabeled EHR notes. This generated our BERT-based model trained using 1.5 million electronic health record notes (EhrBERT). We then further fine-tuned EhrBERT, BioBERT, and BERT on three annotated corpora for biomedical and clinical entity normalization: the Medication, Indication, and Adverse Drug Events (MADE) 1.0 corpus, the National Center for Biotechnology Information (NCBI) disease corpus, and the Chemical-Disease Relations (CDR) corpus. We compared our models with two state-of-the-art normalization systems, namely MetaMap and disease name normalization (DNorm). Results: EhrBERT achieved 40.95% F1 in the MADE 1.0 corpus for mapping named entities to the Medical Dictionary for Regulatory Activities and the Systematized Nomenclature of Medicine—Clinical Terms (SNOMED-CT), which have about 380,000 terms. In this corpus, EhrBERT outperformed MetaMap by 2.36% in F1. For the NCBI disease corpus and CDR corpus, EhrBERT also outperformed DNorm by improving the F1 scores from 88.37% and 89.92% to 90.35% and 93.82%, respectively. Compared with BioBERT and BERT, EhrBERT outperformed them on the MADE 1.0 corpus and the CDR corpus. Conclusions: Our work shows that BERT-based models have achieved state-of-the-art performance for biomedical and clinical entity normalization. BERT-based models can be readily fine-tuned to normalize any kind of named entities. %M 31516126 %R 10.2196/14830 %U http://medinform.jmir.org/2019/3/e14830/ %U https://doi.org/10.2196/14830 %U http://www.ncbi.nlm.nih.gov/pubmed/31516126 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 7 %N 8 %P e11966 %T Deep Learning Intervention for Health Care Challenges: Some Biomedical Domain Considerations %A Tobore,Igbe %A Li,Jingzhen %A Yuhang,Liu %A Al-Handarish,Yousef %A Kandwal,Abhishek %A Nie,Zedong %A Wang,Lei %+ Center for Medical Robotics and Minimally Invasive Surgical Devices, Shenzhen Institutes of Advance Technology, Chinese Academy of Sciences, 1068 Xueyuan Avenue, Shenzhen University, Xili Town, Nanshan District, Shenzhen,, China, 86 755 86585213, zd.nie@siat.ac.cn %K machine learning %K deep learning %K big data %K mHealth %K medical imaging %K electronic health record %K biologicals %K biomedical %K ECG %K EEG %K artificial intelligence %D 2019 %7 02.08.2019 %9 Viewpoint %J JMIR Mhealth Uhealth %G English %X The use of deep learning (DL) for the analysis and diagnosis of biomedical and health care problems has received unprecedented attention in the last decade. The technique has recorded a number of achievements for unearthing meaningful features and accomplishing tasks that were hitherto difficult to solve by other methods and human experts. Currently, biological and medical devices, treatment, and applications are capable of generating large volumes of data in the form of images, sounds, text, graphs, and signals creating the concept of big data. The innovation of DL is a developing trend in the wake of big data for data representation and analysis. DL is a type of machine learning algorithm that has deeper (or more) hidden layers of similar function cascaded into the network and has the capability to make meaning from medical big data. Current transformation drivers to achieve personalized health care delivery will be possible with the use of mobile health (mHealth). DL can provide the analysis for the deluge of data generated from mHealth apps. This paper reviews the fundamentals of DL methods and presents a general view of the trends in DL by capturing literature from PubMed and the Institute of Electrical and Electronics Engineers database publications that implement different variants of DL. We highlight the implementation of DL in health care, which we categorize into biological system, electronic health record, medical image, and physiological signals. In addition, we discuss some inherent challenges of DL affecting biomedical and health domain, as well as prospective research directions that focus on improving health management by promoting the application of physiological signals and modern internet technology. %M 31376272 %R 10.2196/11966 %U https://mhealth.jmir.org/2019/8/e11966/ %U https://doi.org/10.2196/11966 %U http://www.ncbi.nlm.nih.gov/pubmed/31376272 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 3 %P e14499 %T Projection Word Embedding Model With Hybrid Sampling Training for Classifying ICD-10-CM Codes: Longitudinal Observational Study %A Lin,Chin %A Lou,Yu-Sheng %A Tsai,Dung-Jang %A Lee,Chia-Cheng %A Hsu,Chia-Jung %A Wu,Ding-Chung %A Wang,Mei-Chuen %A Fang,Wen-Hui %+ Department of Family and Community Medicine, Tri-Service General Hospital, National Defense Medical Center, No. 325, Section 2, Chenggong Road, Neihu District, Taipei, 11490, Taiwan, 886 02 87923100 ext 18448, rumaf.fang@gmail.com %K word embedding %K convolutional neural network %K artificial intelligence %K natural language processing %K electronic health records %D 2019 %7 23.7.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: Most current state-of-the-art models for searching the International Classification of Diseases, Tenth Revision Clinical Modification (ICD-10-CM) codes use word embedding technology to capture useful semantic properties. However, they are limited by the quality of initial word embeddings. Word embedding trained by electronic health records (EHRs) is considered the best, but the vocabulary diversity is limited by previous medical records. Thus, we require a word embedding model that maintains the vocabulary diversity of open internet databases and the medical terminology understanding of EHRs. Moreover, we need to consider the particularity of the disease classification, wherein discharge notes present only positive disease descriptions. Objective: We aimed to propose a projection word2vec model and a hybrid sampling method. In addition, we aimed to conduct a series of experiments to validate the effectiveness of these methods. Methods: We compared the projection word2vec model and traditional word2vec model using two corpora sources: English Wikipedia and PubMed journal abstracts. We used seven published datasets to measure the medical semantic understanding of the word2vec models and used these embeddings to identify the three–character-level ICD-10-CM diagnostic codes in a set of discharge notes. On the basis of embedding technology improvement, we also tried to apply the hybrid sampling method to improve accuracy. The 94,483 labeled discharge notes from the Tri-Service General Hospital of Taipei, Taiwan, from June 1, 2015, to June 30, 2017, were used. To evaluate the model performance, 24,762 discharge notes from July 1, 2017, to December 31, 2017, from the same hospital were used. Moreover, 74,324 additional discharge notes collected from seven other hospitals were tested. The F-measure, which is the major global measure of effectiveness, was adopted. Results: In medical semantic understanding, the original EHR embeddings and PubMed embeddings exhibited superior performance to the original Wikipedia embeddings. After projection training technology was applied, the projection Wikipedia embeddings exhibited an obvious improvement but did not reach the level of original EHR embeddings or PubMed embeddings. In the subsequent ICD-10-CM coding experiment, the model that used both projection PubMed and Wikipedia embeddings had the highest testing mean F-measure (0.7362 and 0.6693 in Tri-Service General Hospital and the seven other hospitals, respectively). Moreover, the hybrid sampling method was found to improve the model performance (F-measure=0.7371/0.6698). Conclusions: The word embeddings trained using EHR and PubMed could understand medical semantics better, and the proposed projection word2vec model improved the ability of medical semantics extraction in Wikipedia embeddings. Although the improvement from the projection word2vec model in the real ICD-10-CM coding task was not substantial, the models could effectively handle emerging diseases. The proposed hybrid sampling method enables the model to behave like a human expert. %R 10.2196/14499 %U http://medinform.jmir.org/2019/3/e14499/ %U https://doi.org/10.2196/14499 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 7 %P e13659 %T Artificial Intelligence and the Implementation Challenge %A Shaw,James %A Rudzicz,Frank %A Jamieson,Trevor %A Goldfarb,Avi %+ Women's College Hospital, Institute for Health System Solutions and Virtual Care, 76 Grenville Street, Toronto, ON, M5G2A2, Canada, 1 4163236400, jay.shaw@wchospital.ca %K artificial intelligence %K machine learning %K implementation science %K ethics %D 2019 %7 10.07.2019 %9 Viewpoint %J J Med Internet Res %G English %X Background: Applications of artificial intelligence (AI) in health care have garnered much attention in recent years, but the implementation issues posed by AI have not been substantially addressed. Objective: In this paper, we have focused on machine learning (ML) as a form of AI and have provided a framework for thinking about use cases of ML in health care. We have structured our discussion of challenges in the implementation of ML in comparison with other technologies using the framework of Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies (NASSS). Methods: After providing an overview of AI technology, we describe use cases of ML as falling into the categories of decision support and automation. We suggest these use cases apply to clinical, operational, and epidemiological tasks and that the primary function of ML in health care in the near term will be decision support. We then outline unique implementation issues posed by ML initiatives in the categories addressed by the NASSS framework, specifically including meaningful decision support, explainability, privacy, consent, algorithmic bias, security, scalability, the role of corporations, and the changing nature of health care work. Results: Ultimately, we suggest that the future of ML in health care remains positive but uncertain, as support from patients, the public, and a wide range of health care stakeholders is necessary to enable its meaningful implementation. Conclusions: If the implementation science community is to facilitate the adoption of ML in ways that stand to generate widespread benefits, the issues raised in this paper will require substantial attention in the coming years. %M 31293245 %R 10.2196/13659 %U https://www.jmir.org/2019/7/e13659/ %U https://doi.org/10.2196/13659 %U http://www.ncbi.nlm.nih.gov/pubmed/31293245 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 7 %P e13664 %T Reducing Patient Loneliness With Artificial Agents: Design Insights From Evolutionary Neuropsychiatry %A Loveys,Kate %A Fricchione,Gregory %A Kolappa,Kavitha %A Sagar,Mark %A Broadbent,Elizabeth %+ Department of Psychological Medicine, The University of Auckland, Auckland City Hospital, Level 12 Support Building, 85 Park Road, Grafton, Auckland, 1023, New Zealand, 64 9 373 7599 ext 84340, k.loveys@auckland.ac.nz %K loneliness %K neuropsychiatry %K biological evolution %K psychological bonding %K interpersonal relations %K artificial intelligence %K social support %K eHealth %D 2019 %7 08.07.2019 %9 Viewpoint %J J Med Internet Res %G English %X Loneliness is a growing public health issue that substantially increases the risk of morbidity and mortality. Artificial agents, such as robots, embodied conversational agents, and chatbots, present an innovation in care delivery and have been shown to reduce patient loneliness by providing social support. However, similar to doctor and patient relationships, the quality of a patient’s relationship with an artificial agent can impact support effectiveness as well as care engagement. Incorporating mammalian attachment-building behavior in neural network processing as part of an agent’s capabilities may improve relationship quality and engagement between patients and artificial agents. We encourage developers of artificial agents intended to relieve patient loneliness to incorporate design insights from evolutionary neuropsychiatry. %M 31287067 %R 10.2196/13664 %U https://www.jmir.org/2019/7/e13664/ %U https://doi.org/10.2196/13664 %U http://www.ncbi.nlm.nih.gov/pubmed/31287067 %0 Journal Article %@ 2369-3762 %I JMIR Publications %V 5 %N 1 %P e13930 %T Applications and Challenges of Implementing Artificial Intelligence in Medical Education: Integrative Review %A Chan,Kai Siang %A Zary,Nabil %+ Mohammed Bin Rashid University of Medicine and Health Sciences, Building 14, Dubai Healthcare City, PO Box 505055, Dubai,, United Arab Emirates, 971 83571846, nabil.zary@icloud.com %K medical education %K evaluation of AIED systems %K real world applications of AIED systems %K artificial intelligence %D 2019 %7 15.6.2019 %9 Review %J JMIR Med Educ %G English %X Background: Since the advent of artificial intelligence (AI) in 1955, the applications of AI have increased over the years within a rapidly changing digital landscape where public expectations are on the rise, fed by social media, industry leaders, and medical practitioners. However, there has been little interest in AI in medical education until the last two decades, with only a recent increase in the number of publications and citations in the field. To our knowledge, thus far, a limited number of articles have discussed or reviewed the current use of AI in medical education. Objective: This study aims to review the current applications of AI in medical education as well as the challenges of implementing AI in medical education. Methods: Medline (Ovid), EBSCOhost Education Resources Information Center (ERIC) and Education Source, and Web of Science were searched with explicit inclusion and exclusion criteria. Full text of the selected articles was analyzed using the Extension of Technology Acceptance Model and the Diffusions of Innovations theory. Data were subsequently pooled together and analyzed quantitatively. Results: A total of 37 articles were identified. Three primary uses of AI in medical education were identified: learning support (n=32), assessment of students’ learning (n=4), and curriculum review (n=1). The main reasons for use of AI are its ability to provide feedback and a guided learning pathway and to decrease costs. Subgroup analysis revealed that medical undergraduates are the primary target audience for AI use. In addition, 34 articles described the challenges of AI implementation in medical education; two main reasons were identified: difficulty in assessing the effectiveness of AI in medical education and technical challenges while developing AI applications. Conclusions: The primary use of AI in medical education was for learning support mainly due to its ability to provide individualized feedback. Little emphasis was placed on curriculum review and assessment of students’ learning due to the lack of digitalization and sensitive nature of examinations, respectively. Big data manipulation also warrants the need to ensure data integrity. Methodological improvements are required to increase AI adoption by addressing the technical difficulties of creating an AI application and using novel methods to assess the effectiveness of AI. To better integrate AI into the medical profession, measures should be taken to introduce AI into the medical school curriculum for medical professionals to better understand AI algorithms and maximize its use. %M 31199295 %R 10.2196/13930 %U http://mededu.jmir.org/2019/1/e13930/ %U https://doi.org/10.2196/13930 %U http://www.ncbi.nlm.nih.gov/pubmed/31199295 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 5 %P e13216 %T Your Robot Therapist Will See You Now: Ethical Implications of Embodied Artificial Intelligence in Psychiatry, Psychology, and Psychotherapy %A Fiske,Amelia %A Henningsen,Peter %A Buyx,Alena %+ Institute for History and Ethics of Medicine, Technical University of Munich School of Medicine, Technical University of Munich, Ismaninger Straße 22, Munich, 81675, Germany, 49 8941404041, a.fiske@tum.de %K artificial intelligence %K robotics %K ethics %K psychiatry %K psychology %K psychotherapy %K medicine %D 2019 %7 09.05.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Research in embodied artificial intelligence (AI) has increasing clinical relevance for therapeutic applications in mental health services. With innovations ranging from ‘virtual psychotherapists’ to social robots in dementia care and autism disorder, to robots for sexual disorders, artificially intelligent virtual and robotic agents are increasingly taking on high-level therapeutic interventions that used to be offered exclusively by highly trained, skilled health professionals. In order to enable responsible clinical implementation, ethical and social implications of the increasing use of embodied AI in mental health need to be identified and addressed. Objective: This paper assesses the ethical and social implications of translating embodied AI applications into mental health care across the fields of Psychiatry, Psychology and Psychotherapy. Building on this analysis, it develops a set of preliminary recommendations on how to address ethical and social challenges in current and future applications of embodied AI. Methods: Based on a thematic literature search and established principles of medical ethics, an analysis of the ethical and social aspects of currently embodied AI applications was conducted across the fields of Psychiatry, Psychology, and Psychotherapy. To enable a comprehensive evaluation, the analysis was structured around the following three steps: assessment of potential benefits; analysis of overarching ethical issues and concerns; discussion of specific ethical and social issues of the interventions. Results: From an ethical perspective, important benefits of embodied AI applications in mental health include new modes of treatment, opportunities to engage hard-to-reach populations, better patient response, and freeing up time for physicians. Overarching ethical issues and concerns include: harm prevention and various questions of data ethics; a lack of guidance on development of AI applications, their clinical integration and training of health professionals; ‘gaps’ in ethical and regulatory frameworks; the potential for misuse including using the technologies to replace established services, thereby potentially exacerbating existing health inequalities. Specific challenges identified and discussed in the application of embodied AI include: matters of risk-assessment, referrals, and supervision; the need to respect and protect patient autonomy; the role of non-human therapy; transparency in the use of algorithms; and specific concerns regarding long-term effects of these applications on understandings of illness and the human condition. Conclusions: We argue that embodied AI is a promising approach across the field of mental health; however, further research is needed to address the broader ethical and societal concerns of these technologies to negotiate best research and medical practices in innovative mental health care. We conclude by indicating areas of future research and developing recommendations for high-priority areas in need of concrete ethical guidance. %M 31094356 %R 10.2196/13216 %U https://www.jmir.org/2019/5/e13216/ %U https://doi.org/10.2196/13216 %U http://www.ncbi.nlm.nih.gov/pubmed/31094356 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 5 %P e11030 %T Data-Driven Blood Glucose Pattern Classification and Anomalies Detection: Machine-Learning Applications in Type 1 Diabetes %A Woldaregay,Ashenafi Zebene %A Årsand,Eirik %A Botsis,Taxiarchis %A Albers,David %A Mamykina,Lena %A Hartvigsen,Gunnar %+ Department of Computer Science, University of Tromsø – The Arctic University of Norway, Realfagbygget, Hansine Hansens vei 54, Tromsø,, Norway, 47 77646444, ashenafi.z.woldaregay@uit.no %K type 1 diabetes %K blood glucose dynamics %K anomalies detection %K machine learning %D 2019 %7 01.05.2019 %9 Review %J J Med Internet Res %G English %X Background: Diabetes mellitus is a chronic metabolic disorder that results in abnormal blood glucose (BG) regulations. The BG level is preferably maintained close to normality through self-management practices, which involves actively tracking BG levels and taking proper actions including adjusting diet and insulin medications. BG anomalies could be defined as any undesirable reading because of either a precisely known reason (normal cause variation) or an unknown reason (special cause variation) to the patient. Recently, machine-learning applications have been widely introduced within diabetes research in general and BG anomaly detection in particular. However, irrespective of their expanding and increasing popularity, there is a lack of up-to-date reviews that materialize the current trends in modeling options and strategies for BG anomaly classification and detection in people with diabetes. Objective: This review aimed to identify, assess, and analyze the state-of-the-art machine-learning strategies and their hybrid systems focusing on BG anomaly classification and detection including glycemic variability (GV), hyperglycemia, and hypoglycemia in type 1 diabetes within the context of personalized decision support systems and BG alarm events applications, which are important constituents for optimal diabetes self-management. Methods: A rigorous literature search was conducted between September 1 and October 1, 2017, and October 15 and November 5, 2018, through various Web-based databases. Peer-reviewed journals and articles were considered. Information from the selected literature was extracted based on predefined categories, which were based on previous research and further elaborated through brainstorming. Results: The initial results were vetted using the title, abstract, and keywords and retrieved 496 papers. After a thorough assessment and screening, 47 articles remained, which were critically analyzed. The interrater agreement was measured using a Cohen kappa test, and disagreements were resolved through discussion. The state-of-the-art classes of machine learning have been developed and tested up to the task and achieved promising performance including artificial neural network, support vector machine, decision tree, genetic algorithm, Gaussian process regression, Bayesian neural network, deep belief network, and others. Conclusions: Despite the complexity of BG dynamics, there are many attempts to capture hypoglycemia and hyperglycemia incidences and the extent of an individual’s GV using different approaches. Recently, the advancement of diabetes technologies and continuous accumulation of self-collected health data have paved the way for popularity of machine learning in these tasks. According to the review, most of the identified studies used a theoretical threshold, which suffers from inter- and intrapatient variation. Therefore, future studies should consider the difference among patients and also track its temporal change over time. Moreover, studies should also give more emphasis on the types of inputs used and their associated time lag. Generally, we foresee that these developments might encourage researchers to further develop and test these systems on a large-scale basis. %M 31042157 %R 10.2196/11030 %U https://www.jmir.org/2019/5/e11030/ %U https://doi.org/10.2196/11030 %U http://www.ncbi.nlm.nih.gov/pubmed/31042157 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 2 %P e13445 %T The Use of Artificially Intelligent Self-Diagnosing Digital Platforms by the General Public: Scoping Review %A Aboueid,Stephanie %A Liu,Rebecca H %A Desta,Binyam Negussie %A Chaurasia,Ashok %A Ebrahim,Shanil %+ Applied Health Sciences, University of Waterloo, 200 University Avenue West, Waterloo, ON, N2L 3G5, Canada, 1 6134061899, seaboueid@uwaterloo.ca %K diagnosis %K artificial intelligence %K symptom checkers %K diagnostic self evaluation %K self-care %D 2019 %7 01.05.2019 %9 Review %J JMIR Med Inform %G English %X Background: Self-diagnosis is the process of diagnosing or identifying a medical condition in oneself. Artificially intelligent digital platforms for self-diagnosis are becoming widely available and are used by the general public; however, little is known about the body of knowledge surrounding this technology. Objective: The objectives of this scoping review were to (1) systematically map the extent and nature of the literature and topic areas pertaining to digital platforms that use computerized algorithms to provide users with a list of potential diagnoses and (2) identify key knowledge gaps. Methods: The following databases were searched: PubMed (Medline), Scopus, Association for Computing Machinery Digital Library, Institute of Electrical and Electronics Engineers, Google Scholar, Open Grey, and ProQuest Dissertations and Theses. The search strategy was developed and refined with the assistance of a librarian and consisted of 3 main concepts: (1) self-diagnosis; (2) digital platforms; and (3) public or patients. The search generated 2536 articles from which 217 were duplicates. Following the Tricco et al 2018 checklist, 2 researchers screened the titles and abstracts (n=2316) and full texts (n=104), independently. A total of 19 articles were included for review, and data were retrieved following a data-charting form that was pretested by the research team. Results: The included articles were mainly conducted in the United States (n=10) or the United Kingdom (n=4). Among the articles, topic areas included accuracy or correspondence with a doctor’s diagnosis (n=6), commentaries (n=2), regulation (n=3), sociological (n=2), user experience (n=2), theoretical (n=1), privacy and security (n=1), ethical (n=1), and design (n=1). Individuals who do not have access to health care and perceive to have a stigmatizing condition are more likely to use this technology. The accuracy of this technology varied substantially based on the disease examined and platform used. Women and those with higher education were more likely to choose the right diagnosis out of the potential list of diagnoses. Regulation of this technology is lacking in most parts of the world; however, they are currently under development. Conclusions: There are prominent research gaps in the literature surrounding the use of artificially intelligent self-diagnosing digital platforms. Given the variety of digital platforms and the wide array of diseases they cover, measuring accuracy is cumbersome. More research is needed to understand the user experience and inform regulations. %M 31042151 %R 10.2196/13445 %U http://medinform.jmir.org/2019/2/e13445/ %U https://doi.org/10.2196/13445 %U http://www.ncbi.nlm.nih.gov/pubmed/31042151 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 4 %P e13822 %T Detecting Developmental Delay and Autism Through Machine Learning Models Using Home Videos of Bangladeshi Children: Development and Validation Study %A Tariq,Qandeel %A Fleming,Scott Lanyon %A Schwartz,Jessey Nicole %A Dunlap,Kaitlyn %A Corbin,Conor %A Washington,Peter %A Kalantarian,Haik %A Khan,Naila Z %A Darmstadt,Gary L %A Wall,Dennis Paul %+ Division of Systems Medicine, Department of Pediatrics, Stanford University, 1265 Welch Road, Palo Alto, CA, 94305, United States, 1 6173946031, dpwall@stanford.edu %K autism %K autism spectrum disorder %K machine learning %K developmental delays %K clinical resources %K Bangladesh %K Biomedical Data Science %D 2019 %7 24.04.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Autism spectrum disorder (ASD) is currently diagnosed using qualitative methods that measure between 20-100 behaviors, can span multiple appointments with trained clinicians, and take several hours to complete. In our previous work, we demonstrated the efficacy of machine learning classifiers to accelerate the process by collecting home videos of US-based children, identifying a reduced subset of behavioral features that are scored by untrained raters using a machine learning classifier to determine children’s “risk scores” for autism. We achieved an accuracy of 92% (95% CI 88%-97%) on US videos using a classifier built on five features. Objective: Using videos of Bangladeshi children collected from Dhaka Shishu Children’s Hospital, we aim to scale our pipeline to another culture and other developmental delays, including speech and language conditions. Methods: Although our previously published and validated pipeline and set of classifiers perform reasonably well on Bangladeshi videos (75% accuracy, 95% CI 71%-78%), this work improves on that accuracy through the development and application of a powerful new technique for adaptive aggregation of crowdsourced labels. We enhance both the utility and performance of our model by building two classification layers: The first layer distinguishes between typical and atypical behavior, and the second layer distinguishes between ASD and non-ASD. In each of the layers, we use a unique rater weighting scheme to aggregate classification scores from different raters based on their expertise. We also determine Shapley values for the most important features in the classifier to understand how the classifiers’ process aligns with clinical intuition. Results: Using these techniques, we achieved an accuracy (area under the curve [AUC]) of 76% (SD 3%) and sensitivity of 76% (SD 4%) for identifying atypical children from among developmentally delayed children, and an accuracy (AUC) of 85% (SD 5%) and sensitivity of 76% (SD 6%) for identifying children with ASD from those predicted to have other developmental delays. Conclusions: These results show promise for using a mobile video-based and machine learning–directed approach for early and remote detection of autism in Bangladeshi children. This strategy could provide important resources for developmental health in developing countries with few clinical resources for diagnosis, helping children get access to care at an early age. Future research aimed at extending the application of this approach to identify a range of other conditions and determine the population-level burden of developmental disabilities and impairments will be of high value. %M 31017583 %R 10.2196/13822 %U http://www.jmir.org/2019/4/e13822/ %U https://doi.org/10.2196/13822 %U http://www.ncbi.nlm.nih.gov/pubmed/31017583 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 4 %P e12887 %T Physicians’ Perceptions of Chatbots in Health Care: Cross-Sectional Web-Based Survey %A Palanica,Adam %A Flaschner,Peter %A Thommandram,Anirudh %A Li,Michael %A Fossat,Yan %+ Labs Department, Klick Health, Klick Inc, 175 Bloor St E, Suite 300, Toronto, ON, M4W 3R8, Canada, 1 416 214 4977, apalanica@klick.com %K physician satisfaction %K health care %K telemedicine %K mobile health %K health surveys %D 2019 %7 05.04.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Many potential benefits for the uses of chatbots within the context of health care have been theorized, such as improved patient education and treatment compliance. However, little is known about the perspectives of practicing medical physicians on the use of chatbots in health care, even though these individuals are the traditional benchmark of proper patient care. Objective: This study aimed to investigate the perceptions of physicians regarding the use of health care chatbots, including their benefits, challenges, and risks to patients. Methods: A total of 100 practicing physicians across the United States completed a Web-based, self-report survey to examine their opinions of chatbot technology in health care. Descriptive statistics and frequencies were used to examine the characteristics of participants. Results: A wide variety of positive and negative perspectives were reported on the use of health care chatbots, including the importance to patients for managing their own health and the benefits on physical, psychological, and behavioral health outcomes. More consistent agreement occurred with regard to administrative benefits associated with chatbots; many physicians believed that chatbots would be most beneficial for scheduling doctor appointments (78%, 78/100), locating health clinics (76%, 76/100), or providing medication information (71%, 71/100). Conversely, many physicians believed that chatbots cannot effectively care for all of the patients’ needs (76%, 76/100), cannot display human emotion (72%, 72/100), and cannot provide detailed diagnosis and treatment because of not knowing all of the personal factors associated with the patient (71%, 71/100). Many physicians also stated that health care chatbots could be a risk to patients if they self-diagnose too often (714%, 74/100) and do not accurately understand the diagnoses (74%, 74/100). Conclusions: Physicians believed in both costs and benefits associated with chatbots, depending on the logistics and specific roles of the technology. Chatbots may have a beneficial role to play in health care to support, motivate, and coach patients as well as for streamlining organizational tasks; in essence, chatbots could become a surrogate for nonmedical caregivers. However, concerns remain on the inability of chatbots to comprehend the emotional state of humans as well as in areas where expert medical knowledge and intelligence is required. %M 30950796 %R 10.2196/12887 %U https://www.jmir.org/2019/4/e12887/ %U https://doi.org/10.2196/12887 %U http://www.ncbi.nlm.nih.gov/pubmed/30950796 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 4 %P e12286 %T Applications of Machine Learning in Real-Life Digital Health Interventions: Review of the Literature %A Triantafyllidis,Andreas K %A Tsanas,Athanasios %+ Information Technologies Institute, Centre for Research and Technology Hellas, 6th km Charilaou-Thermi Rd, GR 57001 Thermi, Thessaloniki, 60361, Greece, 30 2310 498100, atriand@gmail.com %K machine learning %K data mining %K artificial intelligence %K digital health %K review %K telemedicine %D 2019 %7 05.04.2019 %9 Review %J J Med Internet Res %G English %X Background: Machine learning has attracted considerable research interest toward developing smart digital health interventions. These interventions have the potential to revolutionize health care and lead to substantial outcomes for patients and medical professionals. Objective: Our objective was to review the literature on applications of machine learning in real-life digital health interventions, aiming to improve the understanding of researchers, clinicians, engineers, and policy makers in developing robust and impactful data-driven interventions in the health care domain. Methods: We searched the PubMed and Scopus bibliographic databases with terms related to machine learning, to identify real-life studies of digital health interventions incorporating machine learning algorithms. We grouped those interventions according to their target (ie, target condition), study design, number of enrolled participants, follow-up duration, primary outcome and whether this had been statistically significant, machine learning algorithms used in the intervention, and outcome of the algorithms (eg, prediction). Results: Our literature search identified 8 interventions incorporating machine learning in a real-life research setting, of which 3 (37%) were evaluated in a randomized controlled trial and 5 (63%) in a pilot or experimental single-group study. The interventions targeted depression prediction and management, speech recognition for people with speech disabilities, self-efficacy for weight loss, detection of changes in biopsychosocial condition of patients with multiple morbidity, stress management, treatment of phantom limb pain, smoking cessation, and personalized nutrition based on glycemic response. The average number of enrolled participants in the studies was 71 (range 8-214), and the average follow-up study duration was 69 days (range 3-180). Of the 8 interventions, 6 (75%) showed statistical significance (at the P=.05 level) in health outcomes. Conclusions: This review found that digital health interventions incorporating machine learning algorithms in real-life studies can be useful and effective. Given the low number of studies identified in this review and that they did not follow a rigorous machine learning evaluation methodology, we urge the research community to conduct further studies in intervention settings following evaluation principles and demonstrating the potential of machine learning in clinical practice. %M 30950797 %R 10.2196/12286 %U https://www.jmir.org/2019/4/e12286/ %U https://doi.org/10.2196/12286 %U http://www.ncbi.nlm.nih.gov/pubmed/30950797 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 8 %N 2 %P e12100 %T Artificial Intelligence in Clinical Health Care Applications: Viewpoint %A van Hartskamp,Michael %A Consoli,Sergio %A Verhaegh,Wim %A Petkovic,Milan %A van de Stolpe,Anja %+ Philips Research, HTC11, p247, High Tech Campus, Eindhoven, 5656AE, Netherlands, 31 612784841, anja.van.de.stolpe@philips.com %K artificial intelligence %K deep learning %K clinical data %K Bayesian modeling %K medical informatics %D 2019 %7 05.04.2019 %9 Viewpoint %J Interact J Med Res %G English %X The idea of artificial intelligence (AI) has a long history. It turned out, however, that reaching intelligence at human levels is more complicated than originally anticipated. Currently, we are experiencing a renewed interest in AI, fueled by an enormous increase in computing power and an even larger increase in data, in combination with improved AI technologies like deep learning. Healthcare is considered the next domain to be revolutionized by artificial intelligence. While AI approaches are excellently suited to develop certain algorithms, for biomedical applications there are specific challenges. We propose six recommendations—the 6Rs—to improve AI projects in the biomedical space, especially clinical health care, and to facilitate communication between AI scientists and medical doctors: (1) Relevant and well-defined clinical question first; (2) Right data (ie, representative and of good quality); (3) Ratio between number of patients and their variables should fit the AI method; (4) Relationship between data and ground truth should be as direct and causal as possible; (5) Regulatory ready; enabling validation; and (6) Right AI method. %M 30950806 %R 10.2196/12100 %U https://www.i-jmr.org/2019/2/e12100/ %U https://doi.org/10.2196/12100 %U http://www.ncbi.nlm.nih.gov/pubmed/30950806 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 3 %P e12422 %T Physician Confidence in Artificial Intelligence: An Online Mobile Survey %A Oh,Songhee %A Kim,Jae Heon %A Choi,Sung-Woo %A Lee,Hee Jeong %A Hong,Jungrak %A Kwon,Soon Hyo %+ Division of Nephrology, Department of Internal Medicine, Soonchunhyang University Hospital, Daesagwanro 59 Youngsangu, Seoul, 04401, Republic of Korea, 82 01034413147, ksoonhyo@schmc.ac.kr %K artificial intelligence %K AI %K awareness %K physicians %D 2019 %7 25.03.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: It is expected that artificial intelligence (AI) will be used extensively in the medical field in the future. Objective: The purpose of this study is to investigate the awareness of AI among Korean doctors and to assess physicians’ attitudes toward the medical application of AI. Methods: We conducted an online survey composed of 11 closed-ended questions using Google Forms. The survey consisted of questions regarding the recognition of and attitudes toward AI, the development direction of AI in medicine, and the possible risks of using AI in the medical field. Results: A total of 669 participants completed the survey. Only 40 (5.9%) answered that they had good familiarity with AI. However, most participants considered AI useful in the medical field (558/669, 83.4% agreement). The advantage of using AI was seen as the ability to analyze vast amounts of high-quality, clinically relevant data in real time. Respondents agreed that the area of medicine in which AI would be most useful is disease diagnosis (558/669, 83.4% agreement). One possible problem cited by the participants was that AI would not be able to assist in unexpected situations owing to inadequate information (196/669, 29.3%). Less than half of the participants(294/669, 43.9%) agreed that AI is diagnostically superior to human doctors. Only 237 (35.4%) answered that they agreed that AI could replace them in their jobs. Conclusions: This study suggests that Korean doctors and medical students have favorable attitudes toward AI in the medical field. The majority of physicians surveyed believed that AI will not replace their roles in the future. %M 30907742 %R 10.2196/12422 %U http://www.jmir.org/2019/3/e12422/ %U https://doi.org/10.2196/12422 %U http://www.ncbi.nlm.nih.gov/pubmed/30907742 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 3 %P e12802 %T Artificial Intelligence and the Future of Primary Care: Exploratory Qualitative Study of UK General Practitioners’ Views %A Blease,Charlotte %A Kaptchuk,Ted J %A Bernstein,Michael H %A Mandl,Kenneth D %A Halamka,John D %A DesRoches,Catherine M %+ General Medicine and Primary Care, Beth Israel Deaconess Medical Center, Harvard Medical School, 330 Brookline Avenue, Boston, MA, MA 02215, United States, 1 617 754 1457, cblease@bidmc.harvard.edu %K artificial intelligence %K attitudes %K future %K general practice %K machine learning %K opinions %K primary care %K qualitative research %K technology %D 2019 %7 20.03.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: The potential for machine learning to disrupt the medical profession is the subject of ongoing debate within biomedical informatics and related fields. Objective: This study aimed to explore general practitioners’ (GPs’) opinions about the potential impact of future technology on key tasks in primary care. Methods: In June 2018, we conducted a Web-based survey of 720 UK GPs’ opinions about the likelihood of future technology to fully replace GPs in performing 6 key primary care tasks, and, if respondents considered replacement for a particular task likely, to estimate how soon the technological capacity might emerge. This study involved qualitative descriptive analysis of written responses (“comments”) to an open-ended question in the survey. Results: Comments were classified into 3 major categories in relation to primary care: (1) limitations of future technology, (2) potential benefits of future technology, and (3) social and ethical concerns. Perceived limitations included the beliefs that communication and empathy are exclusively human competencies; many GPs also considered clinical reasoning and the ability to provide value-based care as necessitating physicians’ judgments. Perceived benefits of technology included expectations about improved efficiencies, in particular with respect to the reduction of administrative burdens on physicians. Social and ethical concerns encompassed multiple, divergent themes including the need to train more doctors to overcome workforce shortfalls and misgivings about the acceptability of future technology to patients. However, some GPs believed that the failure to adopt technological innovations could incur harms to both patients and physicians. Conclusions: This study presents timely information on physicians’ views about the scope of artificial intelligence (AI) in primary care. Overwhelmingly, GPs considered the potential of AI to be limited. These views differ from the predictions of biomedical informaticians. More extensive, stand-alone qualitative work would provide a more in-depth understanding of GPs’ views. %M 30892270 %R 10.2196/12802 %U http://www.jmir.org/2019/3/e12802/ %U https://doi.org/10.2196/12802 %U http://www.ncbi.nlm.nih.gov/pubmed/30892270 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 3 %P e11990 %T Detecting Hypoglycemia Incidents Reported in Patients’ Secure Messages: Using Cost-Sensitive Learning and Oversampling to Reduce Data Imbalance %A Chen,Jinying %A Lalor,John %A Liu,Weisong %A Druhl,Emily %A Granillo,Edgard %A Vimalananda,Varsha G %A Yu,Hong %+ Department of Population and Quantitative Health Sciences, University of Massachusetts Medical School, Albert Sherman Center, 9th Floor, 368 Plantation Street, Worcester, MA, 01605, United States, 1 508 856 6063, jinying.chen@umassmed.edu %K secure messaging %K natural language processing %K hypoglycemia %K supervised machine learning %K imbalanced data %K adverse event detection %K drug-related side effects and adverse reactions %D 2019 %7 11.03.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Improper dosing of medications such as insulin can cause hypoglycemic episodes, which may lead to severe morbidity or even death. Although secure messaging was designed for exchanging nonurgent messages, patients sometimes report hypoglycemia events through secure messaging. Detecting these patient-reported adverse events may help alert clinical teams and enable early corrective actions to improve patient safety. Objective: We aimed to develop a natural language processing system, called HypoDetect (Hypoglycemia Detector), to automatically identify hypoglycemia incidents reported in patients’ secure messages. Methods: An expert in public health annotated 3000 secure message threads between patients with diabetes and US Department of Veterans Affairs clinical teams as containing patient-reported hypoglycemia incidents or not. A physician independently annotated 100 threads randomly selected from this dataset to determine interannotator agreement. We used this dataset to develop and evaluate HypoDetect. HypoDetect incorporates 3 machine learning algorithms widely used for text classification: linear support vector machines, random forest, and logistic regression. We explored different learning features, including new knowledge-driven features. Because only 114 (3.80%) messages were annotated as positive, we investigated cost-sensitive learning and oversampling methods to mitigate the challenge of imbalanced data. Results: The interannotator agreement was Cohen kappa=.976. Using cross-validation, logistic regression with cost-sensitive learning achieved the best performance (area under the receiver operating characteristic curve=0.954, sensitivity=0.693, specificity 0.974, F1 score=0.590). Cost-sensitive learning and the ensembled synthetic minority oversampling technique improved the sensitivity of the baseline systems substantially (by 0.123 to 0.728 absolute gains). Our results show that a variety of features contributed to the best performance of HypoDetect. Conclusions: Despite the challenge of data imbalance, HypoDetect achieved promising results for the task of detecting hypoglycemia incidents from secure messages. The system has a great potential to facilitate early detection and treatment of hypoglycemia. %M 30855231 %R 10.2196/11990 %U http://www.jmir.org/2019/3/e11990/ %U https://doi.org/10.2196/11990 %U http://www.ncbi.nlm.nih.gov/pubmed/30855231 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 1 %P e10788 %T Detection of Bleeding Events in Electronic Health Record Notes Using Convolutional Neural Network Models Enhanced With Recurrent Neural Network Autoencoders: Deep Learning Approach %A Li,Rumeng %A Hu,Baotian %A Liu,Feifan %A Liu,Weisong %A Cunningham,Francesca %A McManus,David D %A Yu,Hong %+ Department of Computer Science, University of Massachusetts Lowell, 1 University Avenue, Lowell, MA, 01854, United States, 1 9789343620, hong_yu@uml.edu %K autoencoder %K BiLSTM %K bleeding %K convolutional neural networks %K electronic health record %D 2019 %7 08.02.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: Bleeding events are common and critical and may cause significant morbidity and mortality. High incidences of bleeding events are associated with cardiovascular disease in patients on anticoagulant therapy. Prompt and accurate detection of bleeding events is essential to prevent serious consequences. As bleeding events are often described in clinical notes, automatic detection of bleeding events from electronic health record (EHR) notes may improve drug-safety surveillance and pharmacovigilance. Objective: We aimed to develop a natural language processing (NLP) system to automatically classify whether an EHR note sentence contains a bleeding event. Methods: We expert annotated 878 EHR notes (76,577 sentences and 562,630 word-tokens) to identify bleeding events at the sentence level. This annotated corpus was used to train and validate our NLP systems. We developed an innovative hybrid convolutional neural network (CNN) and long short-term memory (LSTM) autoencoder (HCLA) model that integrates a CNN architecture with a bidirectional LSTM (BiLSTM) autoencoder model to leverage large unlabeled EHR data. Results: HCLA achieved the best area under the receiver operating characteristic curve (0.957) and F1 score (0.938) to identify whether a sentence contains a bleeding event, thereby surpassing the strong baseline support vector machines and other CNN and autoencoder models. Conclusions: By incorporating a supervised CNN model and a pretrained unsupervised BiLSTM autoencoder, the HCLA achieved high performance in detecting bleeding events. %M 30735140 %R 10.2196/10788 %U http://medinform.jmir.org/2019/1/e10788/ %U https://doi.org/10.2196/10788 %U http://www.ncbi.nlm.nih.gov/pubmed/30735140 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 5 %N 4 %P e64 %T Using Psychological Artificial Intelligence (Tess) to Relieve Symptoms of Depression and Anxiety: Randomized Controlled Trial %A Fulmer,Russell %A Joerin,Angela %A Gentile,Breanna %A Lakerink,Lysanne %A Rauws,Michiel %+ Northwestern University, 633 Clark Street, Evanston, IL, United States, 1 312 609 5300 ext 699, russell.fulmer@northwestern.edu %K artificial intelligence %K mental health services %K depression %K anxiety %K students %D 2018 %7 13.12.2018 %9 Original Paper %J JMIR Ment Health %G English %X Background: Students in need of mental health care face many barriers including cost, location, availability, and stigma. Studies show that computer-assisted therapy and 1 conversational chatbot delivering cognitive behavioral therapy (CBT) offer a less-intensive and more cost-effective alternative for treating depression and anxiety. Although CBT is one of the most effective treatment methods, applying an integrative approach has been linked to equally effective posttreatment improvement. Integrative psychological artificial intelligence (AI) offers a scalable solution as the demand for affordable, convenient, lasting, and secure support grows. Objective: This study aimed to assess the feasibility and efficacy of using an integrative psychological AI, Tess, to reduce self-identified symptoms of depression and anxiety in college students. Methods: In this randomized controlled trial, 75 participants were recruited from 15 universities across the United States. All participants completed Web-based surveys, including the Patient Health Questionnaire (PHQ-9), Generalized Anxiety Disorder Scale (GAD-7), and Positive and Negative Affect Scale (PANAS) at baseline and 2 to 4 weeks later (T2). The 2 test groups consisted of 50 participants in total and were randomized to receive unlimited access to Tess for either 2 weeks (n=24) or 4 weeks (n=26). The information-only control group participants (n=24) received an electronic link to the National Institute of Mental Health’s (NIMH) eBook on depression among college students and were only granted access to Tess after completion of the study. Results: A sample of 74 participants completed this study with 0% attrition from the test group and less than 1% attrition from the control group (1/24). The average age of participants was 22.9 years, with 70% of participants being female (52/74), mostly Asian (37/74, 51%), and white (32/74, 41%). Group 1 received unlimited access to Tess, with daily check-ins for 2 weeks. Group 2 received unlimited access to Tess with biweekly check-ins for 4 weeks. The information-only control group was provided with an electronic link to the NIMH’s eBook. Multivariate analysis of covariance was conducted. We used an alpha level of .05 for all statistical tests. Results revealed a statistically significant difference between the control group and group 1, such that group 1 reported a significant reduction in symptoms of depression as measured by the PHQ-9 (P=.03), whereas those in the control group did not. A statistically significant difference was found between the control group and both test groups 1 and 2 for symptoms of anxiety as measured by the GAD-7. Group 1 (P=.045) and group 2 (P=.02) reported a significant reduction in symptoms of anxiety, whereas the control group did not. A statistically significant difference was found on the PANAS between the control group and group 1 (P=.03) and suggests that Tess did impact scores. Conclusions: This study offers evidence that AI can serve as a cost-effective and accessible therapeutic agent. Although not designed to appropriate the role of a trained therapist, integrative psychological AI emerges as a feasible option for delivering support. Trial Registration: International Standard Randomized Controlled Trial Number: ISRCTN61214172; https://doi.org/10.1186/ISRCTN61214172. %M 30545815 %R 10.2196/mental.9782 %U http://mental.jmir.org/2018/4/e64/ %U https://doi.org/10.2196/mental.9782 %U http://www.ncbi.nlm.nih.gov/pubmed/30545815 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 6 %N 11 %P e198 %T The Perceived Benefits of an Artificial Intelligence–Embedded Mobile App Implementing Evidence-Based Guidelines for the Self-Management of Chronic Neck and Back Pain: Observational Study %A Lo,Wai Leung Ambrose %A Lei,Di %A Li,Le %A Huang,Dong Feng %A Tong,Kin-Fai %+ Guangdong Engineering and Technology Research Center for Rehabilitation Medicine and Translation, Department of Rehabilitation Medicine, The First Affiliated Hospital, Sun Yat-sen University, 58 Zhongshan Second Road, Guangzhou, 510000, China, 86 87332200 ext 8536, ambroselo0726@outlook.com %K low back pain %K neck pain %K mobile app %K exercise therapy %K mHealth %D 2018 %7 26.11.2018 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: Chronic musculoskeletal neck and back pain are disabling conditions among adults. Use of technology has been suggested as an alternative way to increase adherence to exercise therapy, which may improve clinical outcomes. Objective: The aim was to investigate the self-perceived benefits of an artificial intelligence (AI)–embedded mobile app to self-manage chronic neck and back pain. Methods: A total of 161 participants responded to the invitation. The evaluation questionnaire included 14 questions that were intended to explore if using the AI rehabilitation system may (1) increase time spent on therapeutic exercise, (2) affect pain level (assessed by the 0-10 Numerical Pain Rating Scale), and (3) reduce the need for other interventions. Results: An increase in time spent on therapeutic exercise per day was observed. The median Numerical Pain Rating Scale scores were 6 (interquartile range [IQR] 5-8) before and 4 (IQR 3-6) after using the AI-embedded mobile app (95% CI 1.18-1.81). A 3-point reduction was reported by the participants who used the AI-embedded mobile app for more than 6 months. Reduction in the usage of other interventions while using the AI-embedded mobile app was also reported. Conclusions: This study demonstrated the positive self-perceived beneficiary effect of using the AI-embedded mobile app to provide a personalized therapeutic exercise program. The positive results suggest that it at least warrants further study to investigate the physiological effect of the AI-embedded mobile app and how it compares with routine clinical care. %M 30478019 %R 10.2196/mhealth.8127 %U http://mhealth.jmir.org/2018/11/e198/ %U https://doi.org/10.2196/mhealth.8127 %U http://www.ncbi.nlm.nih.gov/pubmed/30478019 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 9 %P e11510 %T Patient and Consumer Safety Risks When Using Conversational Assistants for Medical Information: An Observational Study of Siri, Alexa, and Google Assistant %A Bickmore,Timothy W %A Trinh,Ha %A Olafsson,Stefan %A O'Leary,Teresa K %A Asadi,Reza %A Rickles,Nathaniel M %A Cruz,Ricardo %+ College of Computer and Information Science, Northeastern University, 910-177, 360 Huntington Avenue, Boston, MA, 02115, United States, 1 6173735477, bickmore@ccs.neu.edu %K conversational assistant %K conversational interface %K dialogue system %K medical error %K patient safety %D 2018 %7 04.09.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: Conversational assistants, such as Siri, Alexa, and Google Assistant, are ubiquitous and are beginning to be used as portals for medical services. However, the potential safety issues of using conversational assistants for medical information by patients and consumers are not understood. Objective: To determine the prevalence and nature of the harm that could result from patients or consumers using conversational assistants for medical information. Methods: Participants were given medical problems to pose to Siri, Alexa, or Google Assistant, and asked to determine an action to take based on information from the system. Assignment of tasks and systems were randomized across participants, and participants queried the conversational assistants in their own words, making as many attempts as needed until they either reported an action to take or gave up. Participant-reported actions for each medical task were rated for patient harm using an Agency for Healthcare Research and Quality harm scale. Results: Fifty-four subjects completed the study with a mean age of 42 years (SD 18). Twenty-nine (54%) were female, 31 (57%) Caucasian, and 26 (50%) were college educated. Only 8 (15%) reported using a conversational assistant regularly, while 22 (41%) had never used one, and 24 (44%) had tried one “a few times.“ Forty-four (82%) used computers regularly. Subjects were only able to complete 168 (43%) of their 394 tasks. Of these, 49 (29%) reported actions that could have resulted in some degree of patient harm, including 27 (16%) that could have resulted in death. Conclusions: Reliance on conversational assistants for actionable medical information represents a safety risk for patients and consumers. Patients should be cautioned to not use these technologies for answers to medical questions they intend to act on without further consultation from a health care provider. %M 30181110 %R 10.2196/11510 %U http://www.jmir.org/2018/9/e11510/ %U https://doi.org/10.2196/11510 %U http://www.ncbi.nlm.nih.gov/pubmed/30181110 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 5 %N 3 %P e10454 %T An Embodied Conversational Agent for Unguided Internet-Based Cognitive Behavior Therapy in Preventative Mental Health: Feasibility and Acceptability Pilot Trial %A Suganuma,Shinichiro %A Sakamoto,Daisuke %A Shimoyama,Haruhiko %+ Department of Clinical Psychology, Graduate School of Education, The University of Tokyo, , Tokyo,, Japan, 81 3 5841 8068, sgnm.shin@gmail.com %K embodied conversational agent %K cognitive behavioral therapy %K psychological distress %K mental well‐being %K artificial intelligence technology %D 2018 %7 31.07.2018 %9 Original Paper %J JMIR Ment Health %G English %X Background: Recent years have seen an increase in the use of internet-based cognitive behavioral therapy in the area of mental health. Although lower effectiveness and higher dropout rates of unguided than those of guided internet-based cognitive behavioral therapy remain critical issues, not incurring ongoing human clinical resources makes it highly advantageous. Objective: Current research in psychotherapy, which acknowledges the importance of therapeutic alliance, aims to evaluate the feasibility and acceptability, in terms of mental health, of an application that is embodied with a conversational agent. This application was enabled for use as an internet-based cognitive behavioral therapy preventative mental health measure. Methods: Analysis of the data from the 191 participants of the experimental group with a mean age of 38.07 (SD 10.75) years and the 263 participants of the control group with a mean age of 38.05 (SD 13.45) years using a 2-way factorial analysis of variance (group × time) was performed. Results: There was a significant main effect (P=.02) and interaction for time on the variable of positive mental health (P=.02), and for the treatment group, a significant simple main effect was also found (P=.002). In addition, there was a significant main effect (P=.02) and interaction for time on the variable of negative mental health (P=.005), and for the treatment group, a significant simple main effect was also found (P=.001). Conclusions: This research can be seen to represent a certain level of evidence for the mental health application developed herein, indicating empirically that internet-based cognitive behavioral therapy with the embodied conversational agent can be used in mental health care. In the pilot trial, given the issues related to feasibility and acceptability, it is necessary to pursue higher quality evidence while continuing to further improve the application, based on the findings of the current research. %M 30064969 %R 10.2196/10454 %U http://mental.jmir.org/2018/3/e10454/ %U https://doi.org/10.2196/10454 %U http://www.ncbi.nlm.nih.gov/pubmed/30064969 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 6 %P e10148 %T Towards an Artificially Empathic Conversational Agent for Mental Health Applications: System Design and User Perceptions %A Morris,Robert R %A Kouddous,Kareem %A Kshirsagar,Rohan %A Schueller,Stephen M %+ Koko, 155 Rivington St, New York, NY, 10002, United States, 1 617 851 4967, rob@koko.ai %K conversational agents %K mental health %K empathy %K crowdsourcing %K peer support %D 2018 %7 26.06.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: Conversational agents cannot yet express empathy in nuanced ways that account for the unique circumstances of the user. Agents that possess this faculty could be used to enhance digital mental health interventions. Objective: We sought to design a conversational agent that could express empathic support in ways that might approach, or even match, human capabilities. Another aim was to assess how users might appraise such a system. Methods: Our system used a corpus-based approach to simulate expressed empathy. Responses from an existing pool of online peer support data were repurposed by the agent and presented to the user. Information retrieval techniques and word embeddings were used to select historical responses that best matched a user’s concerns. We collected ratings from 37,169 users to evaluate the system. Additionally, we conducted a controlled experiment (N=1284) to test whether the alleged source of a response (human or machine) might change user perceptions. Results: The majority of responses created by the agent (2986/3770, 79.20%) were deemed acceptable by users. However, users significantly preferred the efforts of their peers (P<.001). This effect was maintained in a controlled study (P=.02), even when the only difference in responses was whether they were framed as coming from a human or a machine. Conclusions: Our system illustrates a novel way for machines to construct nuanced and personalized empathic utterances. However, the design had significant limitations and further research is needed to make this approach viable. Our controlled study suggests that even in ideal conditions, nonhuman agents may struggle to express empathy as well as humans. The ethical implications of empathic agents, as well as their potential iatrogenic effects, are also discussed. %M 29945856 %R 10.2196/10148 %U http://www.jmir.org/2018/6/e10148/ %U https://doi.org/10.2196/10148 %U http://www.ncbi.nlm.nih.gov/pubmed/29945856 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 5 %N 2 %P e32 %T Ethical Issues for Direct-to-Consumer Digital Psychotherapy Apps: Addressing Accountability, Data Protection, and Consent %A Martinez-Martin,Nicole %A Kreitmair,Karola %+ Stanford Center for Biomedical Ethics, 1215 Welch Road, Stanford, CA, 94305, United States, 1 650 723 5760, nicolemz@stanford.edu %K ethics %K ethical issues %K mental health %K technology %K telemedicine %K mHealth %K psychotherapy %D 2018 %7 23.04.2018 %9 Viewpoint %J JMIR Ment Health %G English %X This paper focuses on the ethical challenges presented by direct-to-consumer (DTC) digital psychotherapy services that do not involve oversight by a professional mental health provider. DTC digital psychotherapy services can potentially assist in improving access to mental health care for the many people who would otherwise not have the resources or ability to connect with a therapist. However, the lack of adequate regulation in this area exacerbates concerns over how safety, privacy, accountability, and other ethical obligations to protect an individual in therapy are addressed within these services. In the traditional therapeutic relationship, there are ethical obligations that serve to protect the interests of the client and provide warnings. In contrast, in a DTC therapy app, there are no clear lines of accountability or associated ethical obligations to protect the user seeking mental health services. The types of DTC services that present ethical challenges include apps that use a digital platform to connect users to minimally trained nonprofessional counselors, as well as services that provide counseling steered by artificial intelligence and conversational agents. There is a need for adequate oversight of DTC nonprofessional psychotherapy services and additional empirical research to inform policy that will provide protection to the consumer. %M 29685865 %R 10.2196/mental.9423 %U http://mental.jmir.org/2018/2/e32/ %U https://doi.org/10.2196/mental.9423 %U http://www.ncbi.nlm.nih.gov/pubmed/29685865 %0 Journal Article %@ 2369-6893 %I JMIR Publications %V 3 %N 1 %P e37 %T Feasibility of an Automated System Counselor for Survivors of Sexual Assault %A Howe,Esther %A Pedrelli,Paola %A Morris,Robert %A Nyer,Maren %A Mischoulon,David %A Picard,Rosalind %+ Department of Psychiatry, Massachusetts General Hospital, 6th Floor, 1 Bowdoin Square, Boston, MA,, United States, 1 6176437690, ehowe3@mgh.harvard.edu %K CBT %K web chat %D 2017 %7 22.09.2017 %9 Abstract %J iproc %G English %X Background: Sexual assault (SA) is common and costly to individuals and society, and increases risk of mental health disorders. Stigma and cost of care discourage survivors from seeking help. Norms profiling survivors as heterosexual, cisgendered women dissuade LGBTQIA+ individuals and men from accessing care. Because individuals prefer disclosing sensitive information online rather than in-person, online systems—like instant messaging and chatbots—for counseling may bypass concerns about stigma. These systems’ anonymity may increase disclosure and decrease impression management, the process by which individuals attempt to influence others’ perceptions. Their low cost may expand reach of care. There are no known evidence-based chat platforms for SA survivors. Objective: To examine feasibility of a chat platform with peer and automated system (chatbot) counseling interfaces to provide cognitive reappraisals (a cognitive behavioral therapy technique) to survivors. Methods: Participants are English-speaking, US-based survivors, 18+ years old. Participants are told they will be randomized to chat with a peer or automated system counselor 5 times over 2 weeks. In reality, all participants chat with a peer counselor. Chats employ a modified-for-context evidence-based cognitive reappraisal script developed by Koko, a company offering support services for emotional distress via social networks. At baseline, participants indicate counselor type preference and complete a basic demographic form, the Brief Fear of Negative Evaluation Scale, and self-disclosure items from the International Personality Item Pool. After 5 chats, participants complete questions from the Client Satisfaction Questionnaire (CSQ), Self-Reported Attitudes Toward Agent, and the Working Alliance Inventory. Hypotheses: 1) Online chatting and automated systems will be acceptable and feasible means of delivering cognitive reappraisals to survivors. 2) High impression management (IM≥25) and low self-disclosure (SD≤45) will be associated with preference for an automated system. 3) IM and SD will separately moderate the relationship between counselor assignment and participant satisfaction. Results: Ten participants have completed the study. Recruitment is ongoing. We will enroll 50+ participants by 10/2017 and outline findings at the Connected Health Conference. To date, 70% of participants completed all chats within 24 hours of enrollment, and 60% indicated a pre-chat preference for an automated system, suggesting acceptability of the concept. The post-chat CSQ mean total score of 3.98 on a 5-point Likert scale (1=Poor; 5=Excellent) suggests platform acceptability. Of the 50% reporting high IM, 60% indicated preference for an automated system. Of the 30% reporting low SD, 33% reported preference for an automated system. At recruitment completion, ANOVA analyses will elucidate relationships between IM, SD, and counselor assignment. Correlation and linear regression analyses will show any moderating effect of IM and SD on the relationship between counselor assignment and participant satisfaction. Conclusions: Preliminary results suggest acceptability and feasibility of cognitive reappraisals via chat for survivors, and of the automated system counselor concept. Final results will explore relationships between SD, IM, counselor type, and participant satisfaction to inform the development of new platforms for survivors. %R 10.2196/iproc.8585 %U http://www.iproc.org/2017/1/e37/ %U https://doi.org/10.2196/iproc.8585