Time-frequency-spatial channel attention network for semantic decoding: an exploratory EEG study

Semantic decoding is a crucial approach for investigating the neural mechanisms underlying language processing and representation. Informed by brain-computer interface (BCI) technology, this study investigated methods for decoding semantic information, with an emphasis on the neural representations of semantics in language perception. Due to the limited availability of electroencephalography (EEG) datasets containing Chinese linguistic stimuli, we have specifically designed a semantic task paradigm as a promising attempt to decode language comprehension and expression in patients with aphasia using scalp EEG. This paradigm fully incorporates the processes underlying both speech perception and speech imagery by adopting tasks such as overt speech perception and silent speech imagery. Firstly, Seventeen participants of aphasia patients and healthy subjects were recruited for EEG data collection. Secondly, we constructed a deep learning model termed Time-Frequency-Spatial Channel Attention Network (TFSANet), which processes both time-domain and frequency-domain features to extract key neural signatures associated with semantics. By optimizing the model and employing multidimensional feature extraction mechanisms, we significantly improved the model's ability to decode semantically relevant EEG features. Finally, the experimental results demonstrate the proposed TFSANet could decode semantic information from EEG for ten categories of four-word phrases under an "auditory-guided" paradigm with an accuracy of 60.73% and 75.09% for aphasia patients and healthy subjects respectively.