The Potential and Peril of Artificial Intelligence in the Emergency Department

If you’ve worked in an emergency department (ED)—or watched the hit HBO show The Pitt—you can see how artificial intelligence (AI) might be especially helpful in this field. Emergency physicians have mere minutes to narrow down the dizzying array of possibilities: Does the patient have an immune problem? Cancer? A digestive disorder? A psychiatric condition? AI can offer possible diagnoses or recommend tests and treatments based on patients’ symptoms and medical information.

Emergency clinicians “make thousands of decisions daily, often under extreme time pressures and amid considerable variability...the chaotic backdrop of the ED, replete with constant interruptions and distracting stimuli, exacerbates the difficulty,” wrote a group largely consisting of emergency medicine physicians in a paper [1] that outlined the opportunities and challenges of adopting AI in the ED.

Just as the potential of AI for clinical decision support in the ED is substantial, so are the risks. An error due to incomplete information in a generative AI model or a model-generated hallucination can lead to wasted time or the wrong diagnosis or treatment. For example, a STAT News investigation found that an AI algorithm to predict sepsis in the ED routinely missed signs of sepsis. Part of the problem was that the AI incorrectly inferred from training cases that antibiotic use—prescribed after sepsis is suspected—was a predictor of a sepsis diagnosis, so it sometimes didn’t alert providers to potential sepsis until antibiotics had already been ordered [2].

For now, there is inconsistent and haphazard use of AI in EDs to improve diagnostics, with many individual emergency physicians turning to AI models according to their own risk tolerance, explains Hashim Kareemi, MD, lead author of a scoping review [3] of the use of AI in emergency medicine and an emergency physician at Vancouver General Hospital.

Using AI for Clinical Decision Support

In some cases, emergency physicians are engaging with commercially available generative AI models to bolster diagnostic confidence and address potential blind spots, including rare or infrequently encountered diagnoses [4]. Kareemi says he and his colleagues have occasionally used these models and often found their responses to be accurate.

Still, he doesn’t think ChatGPT should be routinely used in the clinical setting. “Most of these commercially available tools are being trained on all the data that’s on the internet...you run the risk of it generating things based on opinions people have shared on Reddit or Twitter.”

Kareemi does, however, consult OpenEvidence [5], which bases its responses on peer-reviewed evidence and sources its recommendations. For a patient who developed angioedema (swelling of the tissue under the skin) while taking a specific medication, the tool provided alternative medications and doses and the risk that the swelling could recur on the medications, says Kareemi.

He stressed that the physician should verify AI information—for example, by clicking the links retrieved by the AI tool. “When we’re making high-stakes decisions, even something as simple as switching a medication, we need to be able to defend that.”

Frontline Physicians Need Practical Guidance on Using AI

Regulators and health care institutions in both Canada and the United States, such as the College of Physicians and Surgeons of Ontario [6] and the American Medical Association [7], provide limited guidance about the use of large language models in direct patient care. While these guidelines cover general questions about keeping patient data private and secure and acquiring patient consent when using AI, they provide little practical information on how to safely use clinical decision support AI. For example, they don’t explain how much and what type of patient information (outside of protected health information) physicians can enter into a generative AI tool before data breach and identification concerns arise.

In a paper published this year in JMIR Medical Informatics [8], emergency doctors Félix Amiot, MD, and Benoit Potier, MD, wrote that “institutions should implement structured verification processes for AI-suggested diagnoses or treatments” and physicians should be encouraged to use AI models that indicate whether a recommendation is based on clear or uncertain evidence.

While it’s difficult to issue guidance “which could be completely irrelevant tomorrow when a new tool comes out,” Kareemi said hospital leaders and medical schools should teach general AI literacy to health care professionals. This will help them understand both the strengths of AI as well as how to mitigate potential harms in clinical decision-making. “The longer we wait, the more the risk builds as these tools become ubiquitous,” he said.

Electronic Health Record–Based AI Tools

One way to reduce risks is for institutions to integrate specific AI tools that are tested for diagnostic accuracy. For example, Alun Ackery, MD, an emergency physician at St. Michael’s Hospital in Toronto, Canada, cocreated a tool along with neurosurgeon Chris Witiw, MD, that uses information from medical imaging files to help emergency physicians determine which traumatic brain injury patients require an urgent neurosurgeon consult. In the coming months, Ackery and his team will be validating the ASIST-TBI tool in 12 hospitals across Ontario. A previous study found the tool was as accurate as neurosurgeons in predicting when brain injury patients would require surgery [9].

Still, Ackery points out that “integrating software into an electronic health record is fraught with challenges,” and legislation hasn’t kept up with developments in AI. For example, researchers often need to enter into data-sharing agreements with each individual hospital site, even when they have research ethics board approval for a study.

“By utilizing artificial intelligence, we could manage our patients much more efficiently, safely, with better follow up,” Ackery said, but the strict regulatory environment “makes it much more challenging to get the work done that will take us there.”

Similarly, Kerstin de Wit, MD, an emergency medicine professor at Queen’s University, has developed a tool called PE Nudge, which flags patients at risk of a pulmonary embolism, a life-threatening condition that is often misdiagnosed as it presents with vague symptoms like fatigue. The AI tool bases its risk scores on patients’ symptoms, vital signs, and medical history. Noting that EDs have seen unprecedentedly high patient volumes in the last decade and hallway consults are common across Canada due to a lack of beds, “we can’t always fully examine patients,” says de Wit. These factors make the ED “the perfect environment to overlook something like pulmonary embolism,” she says.

While de Wit has validated her tool based on retrospective patient charts, her team hasn’t yet found a hospital willing to commit technical resources to integrate the tool into their electronic health record so it can be tested using real-time patient data. “It would be many months of research to make sure you’re identifying the right things in the medical record. There would be many versions to improve upon,” she said.

AI Tools Can Increase Work Interruptions

Another reason for hesitancy, de Wit points out, is that electronic health records “have a track record of implementing warnings that you ignore. They just become annoying.”

Alister Martin, MD, an emergency physician at Massachusetts General Hospital, agrees that individual AI tools that suggest diagnoses or interventions would likely be more distracting than helpful. He described having to close existing algorithm-based pop-ups in a patient’s medical record as “like a sword fight.”

Rather than an AI tool that helps him arrive at a diagnosis, Martin said he’d like an AI summary of a patient’s medical chart. “When I’m in the [ED], and a patient is rolled in through the double doors by the EMS [ambulance] team, I have about three minutes to get caught up on the patient’s chart before I need to start making decisions.” While he says the “technology is ready” for that application, liability issues pose a barrier.

Martin acknowledged that, given time pressures, physicians can miss relevant aspects of patients’ medical histories, but if that results in harm, patients and families can demand accountability from individual physicians and hospitals. “An imperfect human system has accountability tied in,” he said. “If we move to an AI model, where does the accountability lie?”

While incorporation of AI tools into the ED may improve diagnostic accuracy and speed, health care systems will first need to find creative solutions to existing barriers, including liability issues, privacy rules, and technical limitations.

Updated December 18, 2025: An earlier version of this article incorrectly neglected nuance in the example provided regarding an AI algorithm for predicting sepsis. The original example read “...so it didn’t alert providers to potential sepsis…” This has been corrected to “...so it sometimes didn’t alert providers to potential sepsis…”