MECD+: Unlocking Event-Level Causal Graph Discovery for Video Reasoning.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Causal Inference

Background:

• Current video causal reasoning is limited to simple events and question-answering formats.
• It lacks structured analysis for complex, interconnected events in longer videos.

Purpose of the Study:

• Introduce a new task and dataset, Multi-Event Causal Discovery (MECD), for analyzing causal relationships across multiple, chronologically distributed events in videos.
• Develop a framework to generate structured, event-level causal graphs for comprehensive video understanding.

Main Methods:

• Devised a novel framework inspired by Granger Causality, employing an Event Granger Test with mask-based prediction.
• Integrated causal inference techniques like front-door adjustment and counterfactual inference.
• Incorporated context chain reasoning for enhanced robustness.

Main Results:

• The proposed framework effectively reasons complete causal relations in videos.
• Achieved superior performance compared to GPT-4o (5.77% improvement) and VideoChat2 (2.70% improvement).
• Demonstrated the utility of causal relation graphs in downstream tasks like video question answering and event prediction.

Conclusions:

• The MECD framework provides a significant advancement in understanding complex causalities within videos.
• This approach enables more comprehensive and structured video analysis, moving beyond isolated events.