Emotion recognition from multimodal biosignals: supervised and unsupervised machine learning approaches based on EEG
María Consuelo Sáiz-Manzanares1, Raúl Marticorena-Sánchez2
1GIR DATAHES, Consolidated Research Unit N.° 348 JCYL, Health Sciences Department, Universidad de Burgos, Burgos, Spain.
Introduction:
Artificial intelligence (AI) and machine learning (ML) are increasingly applied in psychology, particularly in educational and clinical settings, to analyse complex behavioural and physiological data. However, evidence regarding the capacity of multimodal physiological signals to characterise cognitive and emotional processes in ecologically valid higher education environments remains limited. This study explored the potential of supervised and unsupervised ML approaches to analyse multimodal physiological responses elicited by emotional avatars in a real educational context.
Methods:
A convenience sample of 55 participants, including university students and lecturers, was recruited. After data cleaning, the final sample comprised 48 participants. Electroencephalography (EEG) and galvanic skin response (GSR) signals were recorded while participants observed avatars displaying different emotional states. Event-related potential (ERP)-derived features were extracted from EEG recordings. Supervised learning was conducted using a Random Forest classifier to evaluate the predictive contribution of physiological, subjective (NASA-TLX), and sociodemographic variables in classifying professional category. In addition, unsupervised analyses were performed using k-means clustering and principal component analysis (PCA) to identify latent patterns within the physiological data.
Results:
Sociodemographic variables achieved the highest predictive performance, followed by subjective workload measures, whereas physiological features demonstrated more limited discriminative power. Cluster analysis revealed substantial overlap among participant response profiles, and PCA showed limited separation between groups in the reduced-dimensional space. These findings suggest that physiological responses associated with emotional processing are better represented as continuous patterns of variability rather than as clearly differentiated categories.
Discussion:
The findings highlight both the opportunities and limitations of applying ML techniques to multimodal physiological data in ecologically valid educational settings. Although physiological measures alone showed limited predictive capacity, they provided complementary information regarding cognitive and emotional processes. Furthermore, the study proposes a dashboard-based framework for the automated integration, analysis, and visualisation of multimodal data, supporting future developments in educational psychology, affective computing, and precision psychology. Overall, the results underscore the need for larger samples and more advanced multimodal modelling strategies to improve the interpretability and predictive value of physiological signals in real-world contexts.
