More Than a Signature: Voice as a Clinical Biomarker

Our voice is our signature. In the absence of our image, our voice admits us into buildings, provides reassurance to loved ones far away, and reveals our emotional state to those closest to us. It can even alert physicians to the presence of diseases such as depression and neurodegenerative conditions like Parkinson or Alzheimer disease. Can AI-powered technology harness the clues hidden within the human voice to assist clinicians in identifying disease much earlier in its trajectory—potentially flagging vocal changes too subtle for even the experts to detect?

It’s an ambitious task—translating a voice recorded on a smartphone into improved patient care will require much more than an accurate algorithm. The human voice is complex; voice technologies need to account for acoustic parameters such as pitch, shimmer, and jitter. Beyond physics, there are also biological, regional, and environmental factors to consider, like sex, language, accent, smoking history, and even time of day—the list is endless.

Voice Is a Signal, Not a Diagnosis

Voice analysis is not intended to replace clinical diagnosis but rather to help clinicians screen and follow patients in real time and at scale. Potential benefits include the early diagnosis of disease and the continuous remote monitoring of chronic conditions—all done by an app on a smartphone.

One of the main applications of voice biomarker technology is screening for neurodegenerative disorders in older people. Hugo Botha, MBChB, is a neurologist and codirector of the Speech Innovation Group in Neurology, Artificial Intelligence and Linguistics (SIGNAL) Laboratory at the Mayo Clinic in Rochester, Minnesota. Botha points out that “vocal biomarkers are not new conceptually,” continuing that, “voice has long been used clinically by [speech pathologists and neurologists].” The power of voice biomarker technology, he says is, “moving voice out of the clinic, and onto some sort of device.”

Botha and colleagues are using machine-learning approaches to assist with the early diagnosis and remote monitoring of motor speech disorders (MSDs) like amyotrophic lateral sclerosis (ALS), aphasia, and progressive apraxia of speech. It’s early days, but the results are promising. The SIGNAL team recently submitted a yet-to-be-reviewed preprint examining the accuracy of AI models in detecting MSDs in humans. One model, HuBERT, stood out: it had a 95.1% accuracy in identifying patients with an MSD, though it was less reliable when distinguishing among different MSDs.

Beyond the Brain

It’s not only neurological conditions that affect our voice—cardiovascular disease and diabetes might be reflected in speech long before they present in the arteries and blood. Guy Fagherazzi, PhD, is the director of the Deep Digital Phenotyping Research Unit at the Luxembourg Institute of Health. Fagherazzi, an epidemiologist, incorporates voice and other human metrics into a concept he calls deep digital phenotyping. This involves combining all clinical, biological, imaging, and “omics” data—large-scale genetic and molecular data—with voice to create a digital twin—a virtual patient that can be personalized to predetermine optimal therapeutic strategies before they’re prescribed to the real patient.

‎

When it comes to diagnosis, Fagherazzi says “There is no magic voice biomarker,” emphasizing that, “voice has to be combined with other dimensions of [clinical] data.” With diabetes, Fagherazzi’s research has shown that machine learning algorithms can detect subtle acoustic patterns arising from vocal changes in the larynx that are not otherwise detectable. His Colive Voice study correctly categorized 71% of men and 66% of women as having type 2 diabetes, with higher accuracy in older populations.

The potential benefits of voice technology extend beyond screening; Fagherazzi is hopeful that future low-cost, noninvasive prescreening tools on smartphones will allow people living with diabetes to spend less time worrying about their blood sugar and “time in range” glycemic markers to spend more time in happiness.

Big Data

Independent, regional data-gathering initiatives are beginning to build what might become a universal dataset for clinical voice analysis. In Israel, the Weizmann Institute’s Human Phenotype Project - Voice has collected over 7000 voice recordings as part of a deep phenotyping cohort. The results, in the process of being peer reviewed, indicate that recordings of people counting to 30 in Hebrew may have predictive value for respiratory and sleep disorders in men and anxiety in women.

Given the multitude of variables involved, the experts agree that more data is key for improving the universal accuracy of these applications in other groups. Fagherazzi’s team plans to install voice-recording booths in Luxembourg hospitals to collect clinically validated data at scale. “We want to position voice as a new digital clinical endpoint for clinical research,” he says. “That’s the next step. And maybe once we reach this clinical validation, we might develop better tools that could be used for remote patient monitoring or even diagnosis... but we are not there yet.”

What does the road ahead look like for this technology?

Real-World Validation

Creating large databases of high-quality voice samples recorded in a laboratory is the first step in producing a clinically useful tool in health care—but that is not where the challenges end.

Botha has outlined three challenges that must be addressed before voice-based MSD technology becomes useful in the clinic:

1. Robustness: Can accuracy be maintained outside of a controlled laboratory setting? For example, on recordings from a smartphone in an airport, or a mall, or with a dog barking outside.
2. Validation: Is the technology universally accurate? Does it account for people with differing languages or accents?
3. Clinical usefulness: Does it improve patient care? The technology should flag potential disorders earlier in the disease progression to initiate more effective therapies and better clinical outcomes.

An Expert in Your Pocket

To date, the US Food and Drug Administration has not approved any voice technology, but several, including apps for amyotrophic lateral sclerosis (ALS), depression/anxiety, Alzheimer disease, and heart failure, are being evaluated in clinical trials.

‎

When asked if he is concerned about anxious patients self-diagnosing complex neurological conditions and appearing in his clinic waving their phone at him, Botha smiles and notes that that was happening before the arrival of voice technology. For him, the main benefit of voice biomarkers is timely access to care, an advantage that outweighs everything else. He points out that many people wait several years before advancing neurological symptoms motivate them to seek medical attention from a specialist. With an early-warning device in everyone’s pocket, he hopes that earlier interventions will produce better health outcomes for us all.

Our voice could be more than our signature. With rapidly advancing machine learning technologies trained on large diverse datasets with robust real-world validation, our voice could become the canary in our health coal mine, vocalizing impending health problems while there is still time to correct them.