Transformer-Empowered Invariant Grounding for Video Question Answering

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Video Question Answering (VideoQA) requires understanding scene-question semantics.
• Current models often rely on spurious correlations, hindering reliable reasoning.
• Empirical Risk Minimization (ERM) in VideoQA models exploits irrelevant scene-answer correlations.

Purpose of the Study:

• To propose a causal learning framework for VideoQA.
• To improve the reasoning ability of VideoQA models by addressing spurious correlations.
• To develop a method that grounds question-critical scenes with invariant causal relations.

Main Methods:

• Introduced Invariant Grounding for VideoQA (IGV), a modal-agnostic framework.
• Developed Transformer-Empowered Invariant Grounding for VideoQA (TIGV) as a practical instantiation.
• Ensured grounding of question-critical scenes with invariant causal relationships across interventions.

Main Results:

• IGV framework significantly improves the reasoning ability of leading VideoQA models.
• TIGV instantiation demonstrates enhanced accuracy and visual explainability.
• The proposed methods show superior generalization ability on four benchmark datasets.

Conclusions:

• Invariant grounding is crucial for robust VideoQA.
• The proposed IGV and TIGV frameworks effectively mitigate spurious correlations.
• This approach enhances both the accuracy and interpretability of VideoQA systems.