DRER: Deep Learning-Based Driver's Real Emotion Recognizer.

Area of Science:

• Intelligent transportation systems
• Affective computing
• Machine learning for driver monitoring

Background:

• Driver emotional state monitoring is crucial for intelligent vehicles.
• Facial expressions alone are insufficient for accurate emotion recognition due to driving constraints.
• Existing methods often fail to capture the true emotional state of drivers.

Purpose of the Study:

• To develop a deep learning-based driver's real emotion recognizer (DRER).
• To improve the accuracy of driver emotion recognition by overcoming limitations of facial expression analysis.
• To integrate physiological signals with visual cues for a comprehensive emotion recognition system.

Main Methods:

• Proposed a deep learning algorithm (DRER) combining facial expression recognition and electrodermal activity.
• Utilized a state-of-the-art convolutional neural network for facial expression analysis.
• Implemented a sensor fusion model integrating facial data with electrodermal activity (EDA) signals.
• Conducted human-in-the-loop experiments to collect and categorize driver emotions.

Main Results:

• The sensor fusion approach significantly increased accuracy.
• Achieved a 114% accuracy increase compared to using only facial expressions.
• Achieved a 146% accuracy increase compared to using only electrodermal activity.
• The proposed DRER method reached 86.8% recognition accuracy for induced emotions during driving.

Conclusions:

• The proposed DRER system effectively recognizes drivers' real emotions by fusing facial and physiological data.
• Sensor fusion significantly enhances emotion recognition accuracy compared to unimodal approaches.
• This method offers a more robust solution for monitoring driver emotional states in intelligent vehicles.