An automated extraction of spectral-temporal and spatial-temporal features of EEG for emotion detection

Emotion is an integral part of human cognitive processes and behaviors. Automatic detection and classification of human emotion has been a goal of applied research. This study presents an approach to detecting emotion from multivariate electroencephalogram (EEG) with signal processing methods applied in the temporal, spectral, and spatial domains. In this work, the noise-assisted multivariate empirical mode decomposition (NA-MEMD) is applied to EEG to extract a set of narrow-band intrinsic mode functions (IMF), upon which spectral analysis and spatial connectivity analysis are performed. Applying Hilbert spectral analysis to those IMFs results in the marginal Hilbert spectrum (MHS). MHS is computed for each EEG channel to obtain the spectral energy of each segment. The spectral energy across multiple EEG channels within the same segment is aggregated while the consecutive frames are stacked to give spectral-temporal feature representation. Again, connectivity analysis is performed at each instant with a non-linear measure named phase locking value (PLV) to construct the connectivity map containing spatial-temporal features. A 2D CNN-BiLSTM is adopted to perform emotion detection with the MHS and the PLV features. On classifying high versus low states in valence, arousal, dominance, and liking, PLV showed better performance than MHS with 97.61%, 96.09%, 96.75%, and 97.23% accuracy, respectively, for DEAP dataset. Meanwhile, the highest accuracy of 94.71% is attained on 4-class task. PLV of high oscillatory IMFs outperforms the reported systems with conventional EEG features.