CRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense.

Area of Science:

• Artificial Intelligence
• Computer Vision

Background:

• Visual Question Answering (VQA) systems require advanced reasoning beyond literal interpretation.
• Integrating commonsense knowledge with visual perception is crucial for sophisticated VQA.

Purpose of the Study:

• To introduce a novel VQA benchmark, Compositional Reasoning on vIsion and Commonsense (CRIC), for evaluating visual commonsense grounding.
• To develop an evaluation metric that assesses both answer correctness and the grounding of commonsense in visual data.

Main Methods:

• Proposed the CRIC benchmark with new question types focusing on compositional reasoning.
• Developed an automatic algorithm to generate question samples using scene graphs and knowledge graphs.
• Introduced an evaluation metric combining answer accuracy and commonsense grounding.

Main Results:

• Current VQA models exhibit limitations in grounding commonsense to specific image regions.
• Joint reasoning on visual information and commonsense remains a significant challenge for existing approaches.
• The CRIC dataset and associated methods provide a new resource for advancing VQA research.

Conclusions:

• Effective visual commonsense reasoning is a key area for future VQA development.
• The CRIC benchmark offers a comprehensive evaluation framework for visual commonsense capabilities.
• Further research is needed to improve models' ability to integrate visual perception with commonsense knowledge.