Semantically-Enhanced Feature Extraction with CLIP and Transformer Networks for Driver Fatigue Detection.

Area of Science:

• Computer Science
• Artificial Intelligence
• Transportation Safety

Background:

• Drowsy driving is a major cause of commercial vehicle accidents.
• Current fatigue detection models struggle with feature extraction and network optimization.
• Deep neural networks are commonly used for driver fatigue analysis from video data.

Purpose of the Study:

• To pioneer the use of CLIP (Contrastive Language-Image Pre-training) for fatigue detection.
• To develop a novel network architecture, CT-Net (CLIP-Transformer Network), for enhanced fatigue analysis.
• To improve the accuracy and efficiency of detecting drowsy driving in commercial vehicles.

Main Methods:

• Utilized the CLIP model for pre-training on driver video data.
• Employed a Transformer architecture to extract sophisticated, long-term temporal features from video sequences.
• Developed the CT-Net (CLIP-Transformer Network) integrating CLIP and Transformer components.

Main Results:

• CT-Net achieved an Area Under the Curve (AUC) of 0.892.
• Demonstrated a 36% accuracy improvement over the CNN-LSTM (Convolutional Neural Network-Long Short-Term Memory) model.
• CLIP pre-training improved AUC by 7% compared to ImageNet pre-training; Transformer enhanced AUC by 4% over LSTM.

Conclusions:

• The CT-Net model represents a state-of-the-art approach to fatigue detection.
• CLIP and Transformer architectures effectively address challenges in feature extraction and temporal dependency modeling.
• The proposed method offers a significant advancement in drowsy driving detection for commercial vehicles.