A new brain-computer interface (BCI) is the first to translate both speech and movement with a single device that reads data from implanted electrodes.

In a recent paper published in Nature Neuroscience, researchers revealed how their new BCI simultaneously communicates verbal and non-verbal information in seconds.

This is a major step forward in BCI research, as human communication relies on body language as much as spoken words to convey meaning. For the first time, one implant marries these two separate elements of communication, offering a breakthrough for individuals with severe paralysis.

A New BCI

The team’s new BCI translates electrical signals from the brain into text using artificial intelligence. The decoded information can then be used to control a digital avatar, guiding both its speech and movements.

The BCI field has made significant strides in recent years, allowing humans to control cursors and robotic arms or even communicate through synthesized speech. However, those projects have generally focused on a single mode of communication, whereas this new system is multifunctional, allowing it to capture more of the rich nuance of human communication.

“This gives us hope for neural prosthesis that will enter day-to-day usage,” said Christian Herff, a computational neuroscientist at Maastricht University in the Netherlands.

Since both are related to communication, it should be no surprise that the neural signals responsible for speech can overlap. That can make the business of separating them for two different functions at the same time extremely challenging. Additionally, when the brain commands the body to speak and move at the same time, the neural activity coordinating those activities can vary from what it would be if only one were occurring at a time.

Collecting Human Neural Data

Two volunteers worked with the researchers to test the new BCI, which involved surgically implanted arrays comprising 253 electrodes on the surface of the cortex. By covering portions of the sensorimotor cortex with so many electrodes, the team captured a large number of signals associated with attempted speech and movement.

Following a brainstem stroke, one volunteer experienced impaired speech and movement capabilities. The team instructed him to wave, shake his hands, clap, and nod. Additionally, they gave him five phrases to attempt to speak silently.

The other volunteer had amyotrophic lateral sclerosis (ALS), a motor neuron disease that causes progressive motor impairment and can lead to loss of speech. Although he could only move his facial muscles, he was capable of typing messages. Researchers gave him ten phrases to imagine vocalizing and ten gestures to imagine performing.

BCI Testing Results

Using the electrode implants, the BCI recorded neural activity generated by each attempt, including when participants attempted movement and speech separately or simultaneously. The team then trained AI models to translate those signals, finding that including data from simultaneous activities improved decoding performance.

With this neural activity mapped, the team then had the volunteers attempt to control a digital avatar using the BCI. Again, testing involved isolated and combined verbal and gestural communication. However, this time, instead of relying exclusively on preselected responses, the participants were asked questions in real time.

Accuracy varied between the volunteers, with the first achieving perfect BCI decoding of the tested movements and speech, while the second recorded 75% speech accuracy and 85% gesture accuracy.

While achieving such promising results for combined speech and gesture signals represents a major step forward in BCI research, more work is needed to continue refining the technology. Higher levels of speech accuracy have been achieved with speech-only devices, and this test was limited to individual sentences rather than continuous speech. Additionally, the device captured gestures rather than steady hand positions.

These limitations were partly intentional, as the team initially simplified its decoding tasks while making the jump to simultaneous actions. Further research could expand the system to recognize movements involving additional parts of the body and enable finer control of the many joints in the hands.

The paper, “Simultaneous Speech and Gesture Decoding for Multimodal Communication in Paralysis,” appeared in Nature Neuroscience on September 14, 2026.

Ryan Whalen covers science and technology for The Debrief. He holds an MA in History and a Master of Library and Information Science with a certificate in Data Science. He can be contacted at ryan@thedebrief.org, and follow him on Twitter @mdntwvlf.

While a patient is fully anesthetized and unresponsive, neurons in the hippocampus continue to process language, distinguish different types of words, and generate neural activity consistent with predicting upcoming words.

Researchers at Baylor College of Medicine directly recorded this neural activity, offering new insight into how unconsciousness works at the cellular level. Their findings, published in Nature, show that hippocampal neurons continue to process aspects of language even when the brain is unconscious.

“Our findings show that the brain is far more active and capable during unconsciousness than previously thought,” said Dr. Sameer Sheth, professor and Cullen Foundation Endowed Chair of neurosurgery and a McNair Scholar at Baylor. “Even when patients are fully anesthetized, their brains continue to analyze the world around them.”

a Rare Look Inside the Anesthetized Hippocampus

Sheth’s team recorded from hundreds of individual neurons in the hippocampus, a region of the brain essential for memory formation. Researchers collected data during epilepsy surgeries, specifically anterior temporal lobectomies, when patients were under general anesthesia and the hippocampus was temporarily accessible. This rare opportunity allowed researchers to directly record hippocampal activity, something scalp EEG cannot.

The researchers used Neuropixels probes, high-density recording devices that can monitor hundreds of neurons simultaneously. This study represents the first use of this technology in the human hippocampus during anesthesia. Seven patients participated, all under propofol anesthesia with consistent sedation levels maintained throughout the recordings.

The Unconscious Brain Still Learns

In the initial test, researchers played patients a repeating tone with occasional unexpected pitches. Hippocampal neurons consistently detected these outlier tones, and their sensitivity increased over approximately ten minutes. Researchers usually observe this kind of adaptive neural response during wakefulness.

To dig deeper, the researchers built a computer model of a recurrent neural network, trained to perform the same tone discrimination task. The model showed the same pattern as the human brain recordings, suggesting that this increased sensitivity to outlier tones can arise from local brain circuits alone, without the need for higher-level conscious processing.

Predicting Words Before They’re Spoken

The researchers also played short stories to anesthetized patients while recording from the same neurons. Hippocampal activity tracked language in real time, distinguishing between nouns, verbs, and adjectives. Neural activity reflected both the current word and prediction of the upcoming word in the sequence. “The brain appears to anticipate what comes next in a story, even without conscious awareness,” Sheth said.

“This kind of predictive coding is something we associate with being awake and attentive, yet it’s happening here in an unconscious state,” added Baylor neurosurgery professor Benjamin Hayden.

What This Means for the Science of Consciousness

The results show that the hippocampus can support neural computations associated with complex cognition even during unconsciousness. This suggests that at least some components of language processing and prediction may not require conscious awareness. Sheth’s team proposes that consciousness does not arise from a single brain region but rather emerges from interactions among multiple brain regions. The hippocampus may process information that later becomes part of awareness.

The researchers noted a parallel with artificial intelligence. Large language models generate text by predicting the next word in a sequence; the anesthetized hippocampus showed a similar pattern of prediction. This similarity may provide neuroscientists and AI researchers with a shared framework for studying language prediction based on biological circuitry.

“Can we use these signals to deploy and run a speech prosthetic for some of the parts of the brain that are damaged by stroke or injury? These are questions that we can now consider in relation to this part of the brain,” said study author and Baylor neurosurgery resident Kalman Katlowitz.

While these results are promising, the team acknowledged the study’s limitations. The research focused only on propofol-based general anesthesia, so the results may not apply to other unconscious states such as natural sleep or coma. Additionally, the researchers examined only the hippocampus, so they cannot yet determine how broadly this type of unconscious processing occurs in other brain regions.

“This work pushes us to rethink what it means to be conscious,” Sheth concluded. “The brain is doing much more behind the scenes than we fully understand.”

Austin Burgess is a writer and researcher with a background in sales, marketing, and data analytics. He holds an MBA, a Bachelor of Science in Business Administration, and a data analytics certification. His work focuses on breaking scientific developments, with an emphasis on emerging biology, cognitive neuroscience, and archaeological discoveries.