Brain-Computer Interface: Applications to Speech Decoding and Synthesis to Augment Communication.

Area of Science:

• Neuroscience
• Biomedical Engineering
• Artificial Intelligence

Background:

• Motor pathway damage (e.g., ALS, stroke) can cause locked-in syndrome (LIS), severely limiting communication.
• Existing augmentative and alternative communication (AAC) devices like eye-tracking may be insufficient for LIS patients.
• Brain-computer interfaces (BCIs) offer a promising alternative for AAC, especially when eye-tracking is slow or unreliable.

Purpose of the Study:

• To review recent advances in brain-computer interfaces (BCIs) for real-time speech synthesis in patients with locked-in syndrome (LIS).
• To explore the neuroanatomical and neurophysiological basis of speech production relevant to BCI development.
• To discuss current neural decoding strategies and vocoder technologies for BCI-driven speech synthesis.

Main Methods:

• Utilizing intracranial electrocorticographic (ECoG) recordings from epilepsy surgery patients to study speech production.
• Analyzing neural decoding strategies, including deep learning models and direct speech unit concatenation.
• Evaluating state-of-the-art vocoders for generating natural-sounding audio waveforms.

Main Results:

• Cortical areas for vocalization and articulation span a large region of the ventral sensorimotor cortex.
• Speech acoustics can be decoded and reconstructed from ECoG recordings with dense electrode arrays.
• BCIs show potential for real-time speech synthesis, surpassing limitations of current AAC.

Conclusions:

• BCIs, particularly those leveraging ECoG, hold significant promise for restoring expressive communication in LIS patients.
• Further research into advanced neural decoding and vocoder technology is crucial for improving BCI speech synthesis.
• Directly synthesizing speech via BCI represents a major advancement in AAC for severe motor impairments.