Sequential Attention-based Detection of Semantic Incongruities from EEG While Listening to Speech

We propose a method with attention-based recurrent neural networks (ARNN) for detecting the semantic incongruities in spoken sentences using single-trial electroencephalogram (EEG) signals. 19 participants listened to sentences, some of which included semantically anomalous words. We recorded their EEG signals while they listened. Although previous detection approaches used a word's explicit onset, we used the EEG signals of the whole regions of each sentence, which made it possible to classify the correctness of the sentences without the onset information of the anomalous words. ARNN achieved 63.5% classification accuracy with a statistical significance above the chance level and also above the performances which includes onset information (50.9%). Our results also demonstrated that the attention weights of the model showed that the predictions depended on the feature vectors that are temporally close to the onsets of the anomalous words.