Optimal multimodal feature combination and classifier selection for music-based EEG signal analysis

Music perception is a fundamental human experience, integral to cognitive and emotional processing, making it a crucial area for neuroscientific investigation. This study examined the neural dynamics underlying music perception and identified optimal methods for classifying mental states using electroencephalography (EEG) signals.

We introduced a novel approach using Indian classical music (ICM) to evoke distinctive cognitive states and identified optimal feature-classifier combinations through statistical analysis of classification outcomes. EEG data were collected as participants listened to three ICM ragas. The extracted features included statistical, covariance, wavelet, fractal, and entropy metrics, capturing temporal, spatiotemporal, spectral, and nonlinear neural dynamics. Classification was performed using algorithms with distinct decision boundaries, including Support Vector Machines, k-nearest neighbors (KNN), perceptron, and tree-based models, each optimized via Bayesian Optimization.

Classification performance varied significantly with feature and classifier selection. Covariance features, paired with KNN, achieved the highest accuracy among individual sets. KNN consistently demonstrated high accuracy and low computation time. Combining specific covariance, wavelet, fractal, and entropy features with KNN achieved a maximum accuracy of 98.65 % in binary and 98.01 % in multiclass classification, indicating that music listening engages shared neural dynamics across stimuli, while each raga also elicits distinct neural patterns.

Music perception involves concurrent rhythmic, arrhythmic, and spatiotemporal coordination across multiple brain regions, and multimodal feature combinations significantly improve classification performance.