S-LSTM-ATT: a hybrid deep learning approach with optimized features for emotion recognition in electroencephalogram

Human emotion recognition using electroencephalograms (EEG) is a critical area of research in human-machine interfaces. Furthermore, EEG data are convoluted and diverse; thus, acquiring consistent results from these signals remains challenging. As such, the authors felt compelled to investigate EEG signals to identify different emotions.

A novel deep learning (DL) model stacked long short-term memory with attention (S-LSTM-ATT) model is proposed for emotion recognition (ER) in EEG signals. Long Short-Term Memory (LSTM) and attention networks effectively handle time-series EEG data and recognise intrinsic connections and patterns. Therefore, the model combined the strengths of the LSTM model and incorporated an attention network to enhance its effectiveness. Optimal features were extracted from the metaheuristic-based firefly optimisation algorithm (FFOA) to identify different emotions efficiently.

The proposed approach recognised emotions in two publicly available standard datasets: SEED and EEG Brainwave. An outstanding accuracy of 97.83% in the SEED and 98.36% in the EEG Brainwave datasets were obtained for three emotion indices: positive, neutral and negative. Aside from accuracy, a comprehensive comparison of the proposed model's precision, recall, F1 score and kappa score was performed to determine the model's applicability. When applied to the SEED and EEG Brainwave datasets, the proposed S-LSTM-ATT achieved superior results to baseline models such as Convolutional Neural Networks (CNN), Gated Recurrent Unit (GRU) and LSTM.

Combining an FFOA-based feature selection (FS) and an S-LSTM-ATT-based classification model demonstrated promising results with high accuracy. Other metrics like precision, recall, F1 score and kappa score proved the suitability of the proposed model for ER in EEG signals.