ViE-Take: A Vision-Driven Multi-Modal Dataset for Exploring the Emotional Landscape in Takeover Safety of Autonomous...

Area of Science:

• Intelligent Transportation Systems
• Human-Computer Interaction
• Affective Computing

Background:

• Autonomous vehicle safety is critical, with driver emotions influencing takeover performance.
• Existing datasets lack emotion-aware data, limiting research in this area.
• New energy vehicles with advanced autopilot systems necessitate understanding driver responses.

Purpose of the Study:

• Introduce ViE-Take, the first vision-driven dataset for emotional landscape in autonomous driving takeovers.
• Enable comprehensive exploration of emotion's impact on driver takeover performance.
• Provide deep learning models for predicting takeover readiness, reaction time, and quality.

Main Methods:

• Developed ViE-Take dataset using multi-source emotion elicitation and multi-modal driver data collection.
• Incorporated multi-dimensional emotion annotations for detailed analysis.
• Trained and validated four deep learning models for takeover performance prediction.

Main Results:

• Emotions exhibit diverse and sometimes counterintuitive effects on takeover performance.
• Social media clips effectively elicit emotions, aiding emotion regulation strategies.
• Vision-based deep learning models demonstrate feasibility and potential for predicting takeover performance.

Conclusions:

• ViE-Take dataset facilitates research into emotion-aware takeover safety in intelligent transportation.
• Emotion recognition and regulation are crucial for enhancing autonomous driving safety.
• Vision-based AI holds significant promise for real-time driver monitoring and safety applications.