Joint Answering and Explanation for Visual Commonsense Reasoning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Visual Commonsense Reasoning (VCR) is an advanced form of Visual Question Answering (VQA) requiring deeper visual comprehension.
• Current VCR methods often process question answering and rationale inference separately, neglecting their intrinsic connection.
• This separation leads to suboptimal performance and limits faithfulness in visual reasoning.

Purpose of the Study:

• To investigate the impact of separate processing on VCR performance, specifically examining language shortcuts and generalization.
• To propose a novel framework that effectively couples question answering and rationale inference in VCR.
• To enhance the overall visual reasoning capabilities of existing VCR models.

Main Methods:

• Empirical studies were conducted to analyze language shortcuts and generalization capabilities in current VCR approaches.
• A plug-and-play knowledge distillation framework was developed to integrate the question answering and rationale inference processes.
• A new bridging branch was introduced within the framework to facilitate information flow between the two processes.

Main Results:

• Empirical findings highlighted limitations in VCR methods that treat question answering and rationale inference independently.
• The proposed framework, when applied to existing VCR baselines, demonstrated consistent and significant performance improvements.
• The effectiveness of coupling the two processes was empirically validated on a benchmark dataset.

Conclusions:

• Coupling question answering and rationale inference is crucial for advancing Visual Commonsense Reasoning.
• The proposed knowledge distillation enhanced framework offers a model-agnostic solution for improving VCR.
• This approach successfully bridges the gap between separate VCR components, leading to more robust visual reasoning.