ADVersa: Abductive Driving Accident Video Understanding

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Traffic accident analysis is crucial for safe driving but hindered by scarce, fragmented data and complex environments.
• Understanding accident causes and near-crash dynamics remains a significant challenge in vision-based research.

Purpose of the Study:

• To present a novel framework, Abductive Driving accident Video understanding (ADVersa), for inferring explanations of near-crash scenes.
• To introduce the Multi-Modal Accident video Understanding (MM-AU) dataset for comprehensive accident video analysis.
• To develop advanced models for multimodal accident video understanding tasks.

Main Methods:

• ADVersa framework for visual past recovery, visual prediction, and accident cause synthesis of near-crash scenes.
• Introduction of the MM-AU dataset with 11,727 videos, aligned text, object boxes, and accident cause texts.
• Development of Abductive CLIP and Contrastive Graph Video Pre-training (CGVP) models utilizing relation-aware cross-modal semantic learning.

Main Results:

• ADVersa demonstrates superior performance over state-of-the-art methods in historical frame recovery, future frame prediction, and accident cause reasoning.
• The proposed models achieve advancements in normal-to-accident video synthesis and accident video editing.
• Validation of the framework's effectiveness across diverse accident video understanding tasks.

Conclusions:

• ADVersa offers a robust solution for multimodal accident video understanding, addressing data scarcity and complexity.
• The MM-AU dataset and proposed models significantly contribute to advancing research in vision-based safe driving.
• This work paves the way for improved accident analysis and prevention through enhanced AI capabilities.