Continuous and discrete decoding of overt speech with scalp electroencephalography (EEG)

Area of Science:

• Neuroscience
• Biomedical Engineering
• Machine Learning

Background:

• Neurological disorders significantly impair speech production, affecting millions and reducing quality of life.
• Existing speech interfaces (e.g., eye-tracking, P300 spellers) are often slow and unnatural for affected individuals.
• Speech brain-computer interfaces (BCIs) offer a promising alternative by directly decoding speech characteristics.

Purpose of the Study:

• To investigate the feasibility of decoding speech features using non-invasive electroencephalography (EEG).
• To explore the application of deep learning models for real-time speech decoding from EEG signals.
• To assess participant-independent decoding performance and identify key neural correlates of speech production.

Main Methods:

• Employed a 63-channel EEG system on nine neurologically intact participants reading aloud.
• Utilized deep learning models (CNNs, RNNs with attention) optimized for minimal parameters and small input windows.
• Performed discrete and continuous speech decoding, frequency sub-band analysis, and perturbation analysis for channel identification.

Main Results:

• Achieved statistically significant participant-independent decoding of discrete and continuous speech features from EEG.
• Identified critical frequency bands (delta, theta, gamma) and distributed neural representations across channels.
• Demonstrated feasibility of common speech neural correlates using Leave-One-Out training, reducing data needs.

Conclusions:

• EEG-based speech decoding is feasible for both discrete and continuous speech features, even with EMG artifacts.
• Optimized deep learning models and analysis techniques advance the development of EEG-enabled speech synthesis.
• This research provides a strong foundation for developing practical EEG-based speech BCIs.