TMU-Net: A Transformer-Based Multimodal Framework with Uncertainty Quantification for Driver Fatigue Detection.

Area of Science:

• Neuroscience
• Biomedical Engineering
• Computer Science

Background:

• Driving fatigue is a major cause of traffic accidents.
• Current automated fatigue detection methods face challenges in accuracy, robustness, and cross-subject generalization.
• Multimodal data fusion offers potential to improve driver fatigue estimation.

Purpose of the Study:

• To propose a novel Multimodal Attention Network (TMU-Net) for precise driver fatigue detection.
• To integrate electroencephalogram (EEG) and electrooculogram (EOG) signals for enhanced fatigue assessment.
• To improve the robustness and practicality of fatigue detection systems.

Main Methods:

• Developed TMU-Net with unimodal feature extraction (causal convolution, ConvSparseAttention, Transformer encoders) and a multimodal fusion module (cross-modal attention, uncertainty-weighted gating).
• Incorporated uncertainty quantification to enhance robustness against noise and individual variability.
• Validated performance on the SEED-VIG dataset using cross-subject testing across 23 subjects.

Main Results:

• TMU-Net demonstrated superior performance stability in cross-subject fatigue detection.
• The network effectively leveraged complementary information from EEG (full-band and five-band features) and EOG signals.
• Attention heatmap visualization confirmed the physiological rationality of the EEG-EOG signal fusion strategy.

Conclusions:

• TMU-Net achieves high-precision driver fatigue detection by effectively fusing EEG and EOG signals.
• The proposed method shows significant improvements in robustness and stability for cross-subject fatigue detection.
• The findings highlight the potential of multimodal signal integration and attention mechanisms for advanced fatigue monitoring systems.