SAFE-Drive: A synergistic adaptive fusion framework for multimodal driver emotion recognition in real driving...

Driver emotion recognition is an important component of driver-state monitoring and has potential value for improving traffic safety. In recent years, multimodal driver emotion recognition has attracted increasing attention. However, many existing studies are conducted in simulated environments or rely on limited modality combinations, which restricts their applicability to complex real-driving conditions. Moreover, physiological and behavioral signals collected during driving are susceptible to environmental noise, and substantial between-subject variability degrades cross-subject recognition performance. To address these challenges, we propose SAFE-Drive, an adaptive multimodal fusion framework for driver emotion recognition in real-driving scenarios. The framework uses collaborative multimodal representation learning and a cross-modal attention mechanism to model interactions among EEG signals, facial video, and EDA signals, allowing complementary representations to be learned in a unified feature space. Meanwhile, an attention-based adaptive fusion strategy dynamically adjusts modality contributions across time windows, reducing the influence of noisy or low-quality modality representations. SAFE-Drive also combines source-domain selection with subject-adaptive training through pretraining and few-shot fine-tuning to mitigate individual differences and improve calibration-assisted cross-subject adaptation. We conduct experiments on a self-collected real-driving dataset and further evaluate its transferability on the public PPB-Emo dataset using the available modalities. SAFE-Drive achieves 80.25% average accuracy and 79.91% macro-averaged F1-score under the few-shot adapted LOSO protocol, supporting the effectiveness of multimodal fusion and limited target-subject calibration for driver emotion recognition in real-driving scenarios.