Decoding Covert Speech From EEG by Functional Areas Spatio-Temporal Transformer

Area of Science:

• Neuroscience
• Cognitive Science
• Signal Processing

Background:

• Decoding covert speech from electroencephalogram (EEG) is difficult due to limited understanding of neural pronunciation mapping and low signal-to-noise ratios.
• Covert speech, the imagination of speaking without audible sound or movement, presents unique challenges for neural decoding.

Purpose of the Study:

• To develop an effective framework for decoding covert speech from EEG signals.
• To investigate the neural mechanisms underlying covert speech production and identify discriminative neural features.

Main Methods:

• Developed a large-scale multi-utterance speech EEG dataset from 57 participants.
• Introduced the Functional Areas Spatio-temporal Transformer (FAST) framework to process EEG signals.
• Utilized transformer architecture for sequence encoding of EEG data.

Main Results:

• Identified distinct and interpretable speech neural features through FAST-generated activation maps.
• Visualized neural activation across frontal and temporal brain regions during covert speech.
• Demonstrated the effectiveness of the FAST framework in speech decoding from EEG.

Conclusions:

• This study provides the first interpretable evidence for speech decoding from EEG.
• The FAST framework offers new insights into the discriminative features of covert speech neural representations.
• The developed dataset and framework pave the way for future research in brain-computer interfaces for speech.