General description: Instant Medical History is a tool that was developed by Primetime Medical Software in 1985 to obtain comprehensive information about the history of present illness while also saving physician time and making documentation more complete. The tool has evolved over the last several decades, and it is still in use today.

Design: patients are invited to select a chief complaint through a web-based portal from home or in a medical office waiting room. They are then presented with a multiple-choice–question set about their symptom severity, duration, timing, context, modifying factors, and associated signs of illness. This information is next submitted to the electronic health record through an application programming interface for review before the patient visit and additionally for inclusion in the physician’s note, if desired [14].

Outcomes measured: the company reports that the tool may save up to 6 minutes per clinical encounter [15,16].

Extent of use: the tool is currently being used in 7 countries by 44,500 physicians. The vendor estimates that it will be used in 80 million visits in 2020 (email communication with Matthew Ferrante, Primetime Medical Software, July 21, 2020).

General description: the HELP System, programmed on the Microsoft Query driver, was described in a 1987 publication by Haug et al [17] titled “A Decision-Driven System to Collect the Patient History.” Informaticists at the University of Utah described a computer-administered history-taking system with decision-driven questions designed to create a differential diagnosis for hospital inpatients with pulmonary disease.

Design: the system used a cognitive model of question selection along with a Bayesian scoring algorithm that led to targeted question selection using modular diagnostic frames within a program called QUERY. The program contained yes-or-no questions for up to 182 symptoms; however, using the decision-driven system, patients in the study were asked to answer a mean of 51 (SD 31) questions. The program’s response report was a list of top 5 differential diagnoses, with accompanying likelihood ranging from 0 to 1.

Outcomes measured: when compared with documented discharge diagnoses in a sample of 27 study participants, the tool’s list of 5 differential diagnoses included the principal discharge diagnosis for 85% of the patients.

Extent of use: the tool was also integrated into the hospital’s HELP hospital information system, which pioneered clinical decision support. A subsequent set of tests with an updated diagnostic system and a modified approach to questioning was tested later and reported in the American Association for Medical Systems and Informatics proceedings, including several refinements to the data collection process. Although the tool is no longer in use within the hospital setting, it formed the basis for a subsequent diagnostic application, Iliad, that has been used in medical education [18].

General description: AIDA is a software package developed by the department of medical informatics at Erasmus University, Rotterdam, The Netherlands. Its capability to automate medical history taking was described in 1987 by Quaak et al [19] in a special issue of Computer Methods and Programs in Biomedicine. The tool’s stated purpose was to elicit a comprehensive history and aid physicians in arriving at an accurate diagnosis.

Design: patients were asked to read questions on a screen and press keys corresponding with their answers. The system contained >400 questions relating to 179 different items. Regarding acute complaints, the patient was asked system by system about whether symptoms existed (using a 7-point scale from never to always). If the patient indicated the presence of a symptom, the system asked more questions about frequency, severity, intensity, duration, onset, and location. The final report was displayed to physicians and patients in a narrative format that was designed to mirror how physicians wrote notes.

Outcomes measured: the investigators studied the tool’s performance compared with the gold standard of a physician interview and found agreement to be 25%. They ultimately found that the tool led to a higher number of diagnostic hypotheses and higher diagnostic certainty compared with physician interview alone. Of note, patients required an average of 66 minutes to complete the computerized interview, but their reports of the experience were favorable (92% rated the tool useful) [20].

Extent of use: the tool was never further developed into a product used in routine care.

General description: ParentLink is a tool designed to obtain information from parents to describe their children’s symptoms in the pediatric emergency department setting. ParentLink, the company, was founded in 1989 and acquired by Blackboard in 2014, and the earliest clinical publication was authored by Porter [21] of Boston Children’s Hospital in 1999.

Design: parents accompanying children with nontraumatic complaints were invited to submit data describing the history of present illness. Parents were asked to independently answer questions on an electronic terminal regarding structured question pathways for fever, respiratory symptoms, or gastrointestinal symptoms. If the reason for the visit was not listed, the parent was invited to select other, and they were then shown a textbox for entering open-ended free text. For structured question pathways, parents were invited to answer prompts such as their children’s activity level, fluid intake, and urine output.

Outcomes measured: parental data entry took a mean time of 5 minutes, and data were not shown to treating physicians. The investigators measured the validity of content entered by parents and found it to be comparable with information documented by physicians with improved sensitivity of parental documentation for hydration status. In a related study from 2002 characterizing the free-response pathway, parents entered between 1 and 142 words to describe the reason for the visit (when presented with a character limit of 2048, which is approximately 350-400 words). Most parents described the chief complaint and elements of the history of present illness, and some asked specific questions and added information about the past medical history. When the parents’ text input and physicians’ histories in the electronic health record were compared, 23% (7/30) of the parents’ entries noted details or observations that were not documented by the physicians [21].

Extent of use: subsequent studies of ParentLink have included pediatric patients with other complaints, including head trauma, ear pain, and dysuria [22].

General description: CIDI-Auto is a computerized version of the World Health Organization’s Composite International Diagnostic Interview (CIDI). The tool’s intended purpose is to allow patients to privately answer a series of questions that lead to the automatic generation of a list of psychiatric differential diagnoses. The computerized version was first described and evaluated by investigators at Canberra Hospital in Canberra, Australia, in 1997 [23].

Design: patients were asked to sit at a computer workstation during an acute psychiatric hospital admission and answer yes-or-no questions about psychiatric symptoms. The core module of the instrument (CIDI-Core) contained 20 major questions and 59 subquestions, which took approximately 75 minutes to administer [24]. Each patient’s responses were organized into a report of diagnosis and symptoms that was given to a physician. The report consisted of a summary of active International Classification of Diseases, Tenth Revision, diagnoses (active in the last 30 days); lifetime diagnoses, which were active >1 month ago; and symptoms in major diagnostic areas.

Outcomes measured: psychiatric physicians (the gold standard) agreed with 50% of the CIDI-Auto current diagnoses and indicated that only 22% of the CIDI-Auto reports provided useful new diagnoses, although 63% helped to clarify diagnoses, and 58% could save clinicians some time. They endorsed the CIDI-Auto as a possible aid to indirect or remote diagnosis where histories would be taken by nonexpert staff. With regard to patients, 94% liked the computerized interview, 83% understood the questions without difficulty, and 60% felt more comfortable with the computerized interview than with a physician. Education and previous computer experience promoted positive attitudes and satisfaction with the computerized interview [25].

Extent of use: the CIDI-Auto has now evolved into the World Health Organization World Mental Health-CIDI Instrument, which is now administered by computer in diverse settings across the world [26].

General description: MEDoctor is a commercially produced symptom checker designed to produce a list of differential diagnoses for users with acute or chronic symptoms [27]. MEDoctor Systems Inc, the developer of the tool, was founded in 1999, and the company’s renewed mission statement since 2017 defines its goal as providing patients access to “actionable medical information...before seeing the clinician” so that patients can make cost-effective decisions about their health.

Design: the MEDoctor tool uses a rule out basis that includes the value of negative symptom answers (no input) to rule out disease probabilities. The engine navigates through >4200 symptoms using Bayesian statistics to produce each interview item based upon numerous factors accumulated, such as sex, onset, and yes-or-no responses. The patient interface consists of drop-down menus and yes-or-no questions to characterize their symptoms. At the end of the process, patients are presented with a list of the top 3 differential diagnoses, and they are offered an opportunity to view a text-based report that displays all completed responses and can be sent to a clinician [28].

Outcomes measured: diagnostic agreement between the tool and vignettes is measured.

Extent of use: The diagnostic tool has been used worldwide for >5 years. As of July 2020, according to the chief executive officer, it has been completed 36,860 times by users in the United States, the United Kingdom, South Africa, and the Philippines (email communication with Charles Kelly, MEDoctor Systems Inc, July 21, 2020).

General description: Clinical Expert Operating System (CLEOS) is a tool created in 2008 by Zakim [29] that is currently being tested in a clinical trial at Karolinska Institutet in Stockholm, Sweden. The tool is designed to facilitate thorough history taking from patients, and it also includes capabilities for decision support of diagnosis, management, and risk stratification for clinicians.

Design: the history-taking program is based on the principles of pathophysiology formalized as software algorithms representing medical knowledge as 450 decision trees. A gating mechanism plus feed-forward and feedback loops enable the tool to perform detailed explorations of any aspects of the history that are significant while also avoiding issues of medical redundancy. The data obtained via CLEOS can be formatted into a narrative summary of the most pertinent history findings similar to that of a physician’s note.

Outcomes measured: in a study of 45 patients published in 2008 in which patients underwent usual care and completed a CLEOS interview, the tool detected 3.5 additional problems per patient, some of which were deemed to be clinically significant, such as unrecognized transient ischemic attacks [29]. Currently, the tool is being used in a clinical trial at Karolinska University Hospital in Stockholm, Sweden, for evaluation of patients with chest pain. After initial triage by a physician in the emergency department, patients are invited to answer questions within CLEOS on a tablet computer. Data about pain in the context of the differential diagnoses for chest pain are contained within 29 decision graphs presenting questions that are a mixture of yes-or-no, multiple-choice, and image-based question types. The arrangement of questions for these patients is based on principles of cardiovascular pathophysiology and includes ratings of symptom severity, nature, pain location and radiation, associated cardiac or vagal symptoms, and precipitators of pain. The primary outcome measure is the successful exclusion of acute coronary syndrome at 7 days (using physician diagnosis as the comparator). Secondary outcomes include the ability to calculate risk scores for acute coronary syndrome; exclusion of acute coronary syndrome for 30 days and 1 year; direct costs and resource use; and patient experience regarding feasibility, acceptance, comprehensibility, and technical aspects such as usability [30].

Extent of use: in the clinical trial, CLEOS is being used at a single hospital: Danderyd University Hospital in Stockholm, Sweden.

General description: Mediktor is a symptom-checker tool developed in 2011 by Teckel Medical and StartUp Health, an IBM affiliate. The developers describe Mediktor as “an interactive tool that can analyze users’ symptoms and evaluate their state of health” [31]. Mediktor is available for consumers to use for free on its website as well as major smartphone and tablet systems in the form of a downloadable mobile app [32,33].

Design: the tool presents users with either multiple-choice questions or conversational-style prompts. After a series of questions and responses, users are provided with an assessment of triage urgency, a summary sheet denoting inputted responses and possible diagnoses, and an opportunity to pay for a telemedicine visit with a licensed clinician. Accessible to patients in >180 supported languages, the tool is powered by artificial intelligence and natural language processing.

Outcomes measured: according to an academic study conducted in Spain, Mediktor’s diagnostic accuracy was tested against the gold standard of physician diagnosis, and its primary list of diagnoses matched in 91% of the cases [36].

Extent of use: the tool has become available through Amazon’s Alexa as a skill that can be enabled for free [34] and through Telegram as a chatbot messaging feature [35]. According to the developer, the tool is currently being used at 3 clinical sites in Europe and the United States with 1.2 million user evaluations in 2019 (email communication with Fabiana Rojas, Mediktor, July 21, 2020).

General description: DocResponse is a clinical workflow, patient intake, and documentation tool launched in 2012 that engages users electronically to schedule visits and enter previsit data about past medical history and acute complaints before their face-to-face encounter with a clinician. The tool was developed by a team of technology experts and physicians from various specialties [37]. The tool’s stated goal is to reduce data entry by front desk reception personnel, medical assistants, and clinicians.

Design: patients are invited to enter data on any hospital-provided or personal smart device. The user can complete consent forms; enter data describing past medical, family, surgical, and social histories; complete the review of systems; and use an assessment tool that populates the history of present illness in the medical record before the patient’s arrival. Clinicians are then provided with clinical decision support, including a preliminary differential diagnosis and treatment recommendation. At the end of the visit, the tool can also generate relevant education materials for the patient to review [38].

Outcomes measured: in a 2015 study, DocResponse was found to be the symptom checker most likely to arrive at the correct principal diagnosis out of a sample of 23 similar tools (although its diagnostic accuracy was 18 out of 36 [50% CI 33%-67%]) [39,40].

Extent of use: the tool is currently being used at >170 clinical sites in many care settings such as urgent care, primary care, orthopedics, pediatrics, gastroenterology, multispecialty, and federally qualified health centers throughout the United States. Per vendor report, the tool was used for >225,000 encounters in 2019 (email communication with Tarek Fahl, MD, DocResponse, July 16, 2020).

General description: Digivey is a self-administered computer-assisted interview tool (delivered on Digivey survey software) that was designed to improve diagnostic accuracy and patient safety.

Design: A 2013 academic publication from Newman-Toker’s research group at Johns Hopkins University School of Medicine described the design of this tool [41]. Digivey delivered adaptive questionnaires with approximately 40 items to emergency department patients to elicit clinical history information about their individual symptom presentations. Participants used one of three electronic devices: mobile kiosk, touch-screen monitor, or laptop computer.

Outcomes measured: all 3 electronic device interfaces were deemed to be usable, there were low rates of user error, and administration required approximately 6 minutes [41].

Extent of use: Newman-Toker has continued to use the system extensively in research studies, including for data collection from patients in an ongoing 5-site multicenter clinical trial that screened >3000 patient encounters using the tool. The Johns Hopkins department of neurology is working to deploy the system in several clinical areas, including via mobile platforms where patients will be able to consistently self-administer their medical data (email communication with David Newman-Toker, MD, July 7, 2020).

General description: PatientTouch is a tool created by PatientSafe Solutions Inc to elicit patients to use an electronic questionnaire to describe their symptoms. In a pilot study published in 2014, investigators at Los Angeles County+USC Medical Center described the tool’s use among adult medically stable patients presenting to the emergency department.

Design: patients were asked to use a handheld touch-screen tablet and complete an electronic questionnaire in either English or Spanish before contact with their physician. First, eligible patients were prompted to select one of six chief complaints: low back pain, upper extremity injury, lower extremity injury, abdominal pain, headache, or motor vehicle collision. Next, they were guided through chief complaint–specific algorithms based in questions provided in multiple-choice or point-and-click format.

Outcomes measured: users were asked to rate their experience, usability, and satisfaction with the technology. Patients reported feeling that the device would help them better communicate with their physician and improve their overall quality of care [42].

Extent of use: the tool is not presently being used at any clinical sites, according to its developer.

General description: OurNotes is a tool developed in 2015 by the OpenNotes movement in which the patient is invited to contribute information to their own ambulatory visit notes [43]. The developers’ stated goal is to enhance patient-clinician communication, support patients’ engagement in their care, and save clinician time.

Design: a pilot study publication described the enrollment of primary care clinicians and patients who were already registered to use an institutional messaging portal. A few days before a scheduled visit, the patient was asked to submit answers to two open-ended free-text questions: “How have you been since your last visit?” “What are the most important things you would like to discuss at your visit?” Before the visit, the patient’s answers to these inquiries were routed to the clinician who could then incorporate them into the visit note. The patient could view the note by logging in to the patient portal after the visit [44].

Outcomes measured: patient experience and clinician workload were assessed in a qualitative study, with patients supporting the idea and clinicians thinking it was possible that their workload could decrease if patients helped to produce their own visit notes [44].

Extent of use: 4 academic medical centers participated in the pilot study, with 160 primary care clinicians and 2500 patients participating between 2018 and 2020. Evaluation of the pilot is underway, and pilot sites are considering expansions of the program [45].

General description: FirstHx is a patient intake tool founded in 2016 and developed by physicians in Toronto, Ontario, Canada, that collects information describing a patient’s symptoms and generates focused medical histories in advance of an acute care visit. The developer’s stated goals include a dedication to improving physician-patient communication, reducing documentation time, mitigating medical error, and improving the quality of care [46].

Design: it is built specifically for use in emergency departments, urgent care, and telemedicine. The tool is designed to use a line of questioning similar to that of physicians and generate a history of present illness report covering >240 presenting complaints in up to 10 supported patient languages.

Outcomes measured: the company’s website reports that patients can complete the 3- to 6-minute digital intake using either their personal smartphone, provided tablet device, or through a dedicated kiosk [47].

Extent of use: the developers state that it is available in the Epic App Store. It has undergone pilot testing at >10 sites, and its estimated use will be 600,000 visits per year (email communication with Mark Benaroia, MD, FirstHx, January 27, 2022).

General description: Automated Evaluation of Gastrointestinal Symptoms (AEGIS) is a tool developed in 2016 by researchers in Los Angeles, California, United States, and Ann Arbor, Michigan, United States, to automatically obtain reports of symptoms from patients in the gastroenterology clinic and transform them into a coherent history of present illness.

Design: patients are invited to answer questions through a web-based portal to characterize symptoms as delineated by the Patient-Reported Outcomes Measurement Information System framework. If a patient reports several symptoms, the AEGIS system prompts the user to select the most bothersome symptom. An algorithm generates a physician-facing report that is designed to look like a physician-generated history of present illness.

Outcomes measured: in 1 peer-reviewed publication, patients underwent both computer-generated history taking and usual care, and blinded ratings compared the quality of both sets of documentation. Computer-generated histories were found to be more complete, more useful, better organized, more succinct, and more comprehensible [48]. Another peer-reviewed publication focused on the AEGIS system’s ability to detect alarm features and found its performance to be superior to physician detection (alarm features detected in 53% vs 27%, respectively) [49].

Extent of use: AEGIS has not been used in routine patient care outside of the aforementioned studies (email communication with Christopher Almario, MD, Cedars-Sinai Medical Center, July 19, 2020).

General description: DCAT, created in 2017 by German research and technology experts at aidminutes GmbH, is an anamnesis tool that facilitates communication between patients and health care providers in primary care settings, with particular attention to refugee care sites in Germany that experienced an influx of Syrian patients. Aidminutes, which is affiliated with the department of general practice at the University Medical Center Göttingen in Göttingen, Germany, refers to the tool as a digital communication assistance tool that aims to improve diagnostic accuracy by improving medical history taking [50].

Design: once patients arrive at the outpatient clinic waiting room, they are given tablet devices that allow them to enter information describing their symptoms. The tool also facilitates entry of data describing past medical history, current medications, allergies, and any psychological comorbid conditions. To facilitate use by patients with limited literacy, the tool is designed to be visually intuitive, and it also includes audio prompts. After patients enter their data, responses are translated into the clinician’s preferred native language and presented as a data synopsis, including alerts regarding red flags in the history that have been discovered [51].

Outcomes measured: use of the tool has been described in peer-reviewed publications, including a recent study demonstrating good usability and acceptance by patients speaking Levantine Arabic, Modern Standard Arabic, Egyptian Arabic, Farsi, Sorani Kurdish, and Turkish [52]. In 1 study, patients successfully completed their assessments in an average time of 13 minutes [53].

Extent of use: the tool was used in approximately 10,000 multilingual visits in urgent care and family medicine clinics during 2021 (email communication with Frank Muller and Andreas Lippke, May 3, 2022).

General description: Quro is a chatbot health assistant created in 2017 by Medius Health (Sydney, New South Wales, Australia) that uses machine learning artificial intelligence to deliver health assessments to users [54]. Patients can access the tool through a desktop web browser or smartphone.

Design: users are invited to answer a set of free-response and multiple-choice prompts generated by the back-end sequential question prediction algorithm using a large-scale clinical knowledge graph to mimic the taking of a medical history. Each new question is predicted based on previous user-chat context. After completion, the patient is provided a list of differential diagnoses with interpretations, nearby health services, recommendations about the urgency of their condition, and a detailed report that displays the answers to their individual set of questions. The tool’s website advertises a built-in medical dictionary of >7 million disease and illness presentation patterns, scored against content sourced from “trusted sources like medical journals” to produce its personalized clinical assessments [55].

Outcomes measured: in an article published by Quro’s developers, the tool’s triage accuracy was assessed using 30 case-based scenarios (10 for emergency care, 10 for general practitioner care, and 10 for self-care) and found to be accurate in 83% (25/30) of the cases [56].

Extent of use: Quro is marketed to, and used by, several health and wellness service providers to engage with, and onboard, patients remotely (email communication with Shameek Ghosh, founder and chief technology officer, September 1, 2020).

General description: Mandy is a primary care conversational-style dialogue system developed in 2017 by a public-private research partnership of computer scientists at the University of Auckland in Auckland, New Zealand, funded by Precision Driven Health and Orion Health, aimed at improving health outcomes through data science. The tool is designed to assist health care staff by automating the patient intake process.

Design: patients interact with the tool by answering the conversational-style prompts with open-ended free-text responses. An analytic engine uses natural language processing to interpret the patient’s text, queries a symptom-to-cause mapper for reasoning about potential diagnostic causes, and then generates further interview questions. Once the system has obtained sufficient information from the patient, it reports a differential diagnosis for the clinician to consider.

Outcomes measured: in a proof-of-concept paper, the developers reported on the application’s question accuracy (ability to generate key follow-up questions after an initial chief complaint) and diagnosis prediction accuracy (ability to generate relevant differential diagnoses after obtaining responses to its questions) by using gold-standard cases from a medical textbook. Out of 6 cases, the tool generated appropriate questions in 5 cases and had case-by-case prediction accuracy ranging from 14% to 100% [57].

General description: Diagnosis and Anamnesis Automated Medical History–Taking Device (DIAANA AMHTD) was developed by Logic-Based Medicine Sàrl in collaboration with Adrien Schwitzguebel of Lausanne University Hospital in Lausanne, Switzerland, to improve diagnostic accuracy through helping physicians to generate more comprehensive differential diagnoses. To date, the tool exclusively addresses musculoskeletal complaints.

Design: in a pilot study published in 2019, the developer and coinvestigators tested DIAANA AMHTD at a teaching hospital in Geneva, Switzerland. Patients were eligible if they were waiting to be seen at an ambulatory clinic for evaluation of musculoskeletal symptoms. Patients in the experimental group were asked to complete a digital form on a touch pad before the resident physician’s evaluation, including questions about specific symptoms and risk factors. Through the completion of an adaptive questionnaire that draws from a data set of 269 questions, DIAANA AMHTD then generated a comprehensive anamnesis summary and a list of top differential diagnoses. The list of differential diagnoses (selected from 126 possibilities) was then presented to medical residents for consideration before the face-to-face evaluation.

Outcomes measured: residents who used the tool were found to be more likely to have included the final diagnosis from the list of initial differential diagnoses than those residents who did not use the tool (75% vs 59%, respectively) [58].

Extent of use: the tool is currently being used regularly by a single physician with 250 patient encounters in 2019 and has also been implemented by a Swiss telemedicine system called Soignez-Moi in Biel, Switzerland (Email communication with Adrien Schwitzguebel, MD, July 2020).