How AI Is Speeding Up the Diagnostic Odyssey for Rare Diseases

The benefits of AI algorithms for rare diseases are already tangible for some people. The tools have helped their doctors diagnose them, opening the door for accessibility accommodations, treatments, trials, and more research into their conditions. If these AI tools scale, they could cut down on the diagnostic odyssey—the almost 5 years (on average) of appointments, medical visits, and frustration it usually takes to diagnose rare diseases.

The Children’s Hospital of Eastern Ontario (CHEO) is helping lead the way with an algorithm called ThinkRare, which scans electronic health records and flags children who may need a referral to a rare disease specialist for more testing.

‎

“The purpose of ThinkRare is to find the patients that aren’t coming to us, that are bouncing around, that are lost in the system,” says Ivan Terekhov, BCom—the director of research informatics, AI, and technology at CHEO Research Institute. ThinkRare has already prompted genetic sequencing and follow-up in a handful of patients, leading to 21 new rare disease diagnoses and a 70% success rate so far. The researchers behind the algorithm are working on rolling out the model to other hospitals.

Boston Children’s Hospital (BCH) is addressing the other end of the diagnostic odyssey with a reasoning model built by OpenAI to help solve cases that stump clinicians. Catherine Brownstein, PhD—scientific director of the genetic investigations arm of the Manton Center for Orphan Disease Research at BCH—says she was skeptical at first.

When they fed the reasoning model rare disease cases where they already knew the diagnosis, it initially seemed to falter. Eventually, it diagnosed 19 out of 20 cases correctly—and ultimately, all 20. “When we looked at the one that it got wrong,” she says, “we were wrong.” The AI had flagged a second diagnosis that clinicians had missed.

At that point, Brownstein and her team thought the model was ready for the real test: helping solve undiagnosed cases. The reasoning model was remarkably successful; its output led to 18 diagnoses among 376 previously unresolved cases.

What Does AI Do That Humans Can’t?

According to a recent report, insufficient physician training might help explain why patients with rare diseases go undiagnosed for so long. The ThinkRare algorithm addresses this gap by incorporating expert-curated criteria to flag rare diseases, which the average doctor might miss, for further genetic testing.

After sequencing, it still takes rare disease specialists a long time to determine a diagnosis, and sometimes, complicated cases go unsolved. Specialists have a limited amount of time to spend with each family and may have many cases they’re unable to revisit.

‎

During her PhD, Brownstein spent 6 years working on one family. “There’s just simply not enough geneticists in this world or scientists in this world to do the thorough, in-depth investigations that are necessary to make all the insights possible,” she says.

The reasoning model used at BCH—designed to reason carefully and show its work—was developed using o3 Deep Research, a publicly accessible model.

When the model identifies a potential genetic mutation that might be causing the symptoms, it presents its chain of reasoning. From there, geneticists take over and run tests to confirm the diagnosis. “But instead of having to go to the sequence and find that variant yourself, it’s presented to you in a way that’s much, much quicker,” says Brownstein.

The insights provided by the model may lead to new discoveries that clinicians wouldn’t otherwise make. It surfaced, for example, a new genetic variant that might be implicated in vitiligo, an autoimmune disease where the body attacks the pigment-producing cells in the skin. “I was speechless,” Brownstein recalls. They now have research underway to investigate whether the mutation is causative. If the research bears out, she hopes it could lead to a new treatment.

Why There Needs to Be a Human in the Loop

These models still require human involvement.

ThinkRare’s algorithm is passive, flagging a few cases out of several hundred thousand children visiting their clinics. The children still need to undergo testing and evaluation by a specialist, since the algorithm doesn’t actually predict what kind of disease the child may have.

And since the algorithm is based on clinical expertise and trained on electronic medical records, there is built-in bias. “I think we’d be naive to say that there’s not data quality issues or there’s worse data quality for those who are not White and those who do come from rural locations,” notes Alexandre White-Brown, MSc—a genetic counselor and project manager of ThinkRare. However, Canada’s diversity is still well represented within the algorithm’s training dataset to help offset these potential issues.

While Brownstein says the reasoning model used at BCH improved rapidly, errors and hallucinations are common with any large language model (LLM). OpenAI’s reasoning model still needs to be used in collaboration with a clinician rather than as a stand-alone tool.

Research into human-AI interaction has also repeatedly shown that experts may defer to the judgment of an AI, even when it’s wrong.

“Previous studies show that learning from evidence is especially difficult when the evidence is poorer and uncertainty is higher, as in the case of patients with rare diseases,” says Aranzazu Viñas, PhD—an assistant professor at the University of the Basque Country. “Therefore, AI’s errors might be still more difficult to detect in the case of rare diseases.”

Her recent study suggests that doctors will trust the judgment of an AI, even when they receive new clinical information that shows it’s wrong. Viñas emphasizes the importance of conducting “ecological research” to see if these errors appear in real-world settings. Beyond that, she says it’s “important to develop strategies and protocols that increase human critical thinking and detection of AI errors.”

The Future of AI for Rare Diseases

BCH is looking toward further study and model validation to measure time and cost savings, clinician effort, false positives, and whether using the model changes care for patients. The team is working with OpenAI to democratize access for other researchers and clinicians.

Meanwhile, although ThinkRare is able to spot the low-hanging fruit, Terekhov says that there are data silos created by information locked away in unstructured reports and doctors’ notes. The next iteration of ThinkRare will incorporate LLMs that can access and interpret this information and flag even more cases. Scaling the model countrywide will require regulators to sign off on it.

White-Brown explains the impact ThinkRare is already having for families. Antony was diagnosed with an ultrarare disease called Chung-Jansen Syndrome at 10 years old, after being flagged by the algorithm. The diagnosis provided an explanation for his symptoms, and testing revealed that the same condition affected his brothers and mother.

“The biggest thing for this family was actually being able to access resources in school that were previously unattainable,” says White-Brown.