Generalizing neural signal-to-text brain-computer interfaces.

Area of Science:

• Neuroscience
• Biomedical Engineering
• Computational Linguistics

Background:

• Brain-Computer Interfaces (BCI) offer potential communication solutions for individuals with neurodegenerative diseases.
• Existing BCIs face challenges in speed, accuracy, and generalizability, limiting practical application.
• Current speech decoding BCIs often rely on limited vocabularies, restricting natural communication.

Purpose of the Study:

• To evaluate the performance of a robust speech decoding system for translating neural signals into text.
• To develop an end-to-end BCI system capable of decoding speech without vocabulary constraints.
• To enhance communication accessibility for patients with impaired speech production.

Main Methods:

• Utilized depth-electrode neural signals associated with overt speech.
• Implemented a Long Short-Term Memory (LSTM) model to decode phoneme probabilities from neural features.
• Employed a particle filtering algorithm with English language priors for temporal smoothing and text output.

Main Results:

• Achieved an average word accuracy of 32% across 6 neurosurgical patients.
• Obtained phoneme-level precision of 46% and recall of 51%.
• Demonstrated significant speed improvements compared to existing BCI approaches.

Conclusions:

• The developed neural signal-to-text model offers a more general approach to speech decoding.
• This BCI system has the potential to significantly improve communication for patients in real-world settings.
• The study highlights the feasibility of unconstrained vocabulary speech decoding using neural signals.