Adaptive Multi-Temporal Fusion and Cross-Modal Adversarial Alignment for Robust Driver Fatigue Detection.

Area of Science:

• Intelligent transportation systems
• Computer vision
• Machine learning

Background:

• Driver fatigue poses significant safety risks, necessitating advanced detection methods.
• Existing systems face challenges with multi-scale temporal dynamics and sensor intrusiveness.
• Non-intrusive, accurate driver fatigue detection is crucial for intelligent cockpit safety.

Purpose of the Study:

• To propose a novel deep learning framework, MTFA-Net, for accurate and non-intrusive driver fatigue detection.
• To address limitations in capturing temporal dynamics and sensor-based intrusiveness in current fatigue detection systems.
• To enhance intelligent cockpit safety through real-time fatigue monitoring.

Main Methods:

• Developed the Multi-Temporal Fusion Attention Network (MTFA-Net) integrating two key modules.
• Implemented a Multi-scale Temporal Adaptive Fusion (MTAF) module for dynamic feature weighting.
• Introduced a Physiological-Behavioral Cross-modal Adversarial Alignment (PBCAA) network for inferring physiological states from facial videos.

Main Results:

• MTFA-Net achieved state-of-the-art accuracy of 92.8% on RLDD and NTHU-DDD datasets.
• The framework demonstrated high interpretability and real-time efficiency.
• Successfully inferred latent physiological states (e.g., heart rate variability) from facial data.

Conclusions:

• MTFA-Net provides a robust and non-intrusive solution for driver fatigue detection.
• The proposed methods effectively handle multi-scale temporal dynamics and reduce sensor intrusiveness.
• This approach significantly advances intelligent cockpit safety systems.