Speech Synthesis from Stereotactic EEG using an Electrode Shaft Dependent Multi-Input Convolutional Neural Network...

Area of Science:

• Neuroscience
• Biomedical Engineering
• Speech Science

Background:

• Neurological disorders can cause speech loss, necessitating alternative communication methods.
• Brain-computer interfaces (BCIs) translate neural activity into speech representations.
• Existing speech BCIs primarily use superficial electrocorticography (ECoG), neglecting deeper brain signals.

Purpose of the Study:

• To adapt a speech decoding pipeline for speech synthesis using stereoelectroencephalography (sEEG) signals.
• To investigate the potential of deeper brain activity for speech BCIs.

Main Methods:

• Utilized a multi-input convolutional neural network to process sEEG data.
• Extracted speech-related neural activity from individual electrode shafts.
• Estimated spectral coefficients to reconstruct audible speech waveforms.

Main Results:

• Achieved high correlations (up to 0.80) between original and reconstructed speech spectrograms.
• Demonstrated significantly above chance-level decoding performance for all patients.
• Showed that sEEG yields comparable speech decoding performance to ECoG.

Conclusions:

• Stereoelectroencephalography (sEEG) is a viable and promising modality for developing advanced speech BCIs.
• Harnessing deeper brain activity with sEEG enhances the potential for restoring speech communication.