STFTransNet: A Transformer Based Spatial Temporal Fusion Network for Enhanced Multimodal Driver Inattention State...

Area of Science:

• Computer Science
• Artificial Intelligence
• Transportation Engineering

Background:

• Driver inattention, including drowsiness and distraction, is a major cause of traffic accidents.
• Existing driver inattention recognition systems struggle with challenges like partial facial occlusion and image degradation due to lighting variations.

Purpose of the Study:

• To propose a novel transformer-based spatial-temporal fusion network (STFTransNet) for robust driver inattention state recognition.
• To enhance the accuracy of driver state recognition in real-world driving scenarios with occluded faces and varying light conditions.

Main Methods:

• Utilized mediapipe face mesh for facial landmark extraction.
• Employed a two-stream cross-attention mechanism (RCN-based) for spatial feature learning from face and body action images.
• Implemented a temporal convolutional network (TCN-based) for temporal feature extraction.
• Fused spatial and temporal features for final driver state classification.

Main Results:

• The proposed STFTransNet achieved superior accuracy compared to existing models on multiple public datasets (NTHU-DDD, StateFarm, YawDD).
• Demonstrated significant performance improvements of 4.56%, 3.48%, and 3.78% over baseline models on respective datasets.
• Effectively addressed performance degradation caused by partial facial occlusion and light blur through multi-modality fusion.

Conclusions:

• STFTransNet offers a promising solution for advanced driver assistance systems, improving safety through accurate inattention detection.
• The spatial-temporal fusion approach effectively handles challenging real-world driving conditions, paving the way for more reliable driver monitoring.