Reducing Vision-Answer Biases for Multiple-Choice VQA.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Multiple-choice visual question answering (VQA) requires deep multimodal understanding and reasoning.
• Existing VQA models often rely on multimodal interaction modules but can suffer from vision-answer bias, hindering performance and generalization.

Purpose of the Study:

• To propose a novel, model-agnostic method to mitigate vision-answer bias in VQA.
• To enhance the robustness and accuracy of VQA models through improved inter-modality reasoning.

Main Methods:

• Introduced the Causality-based Multimodal Interaction Enhancement (CMIE) method.
• CMIE incorporates a causal intervention module to eliminate spurious correlations and a counterfactual interaction learning module for supervised multimodal training.
• Designed as a plug-and-play component for existing VQA architectures.

Main Results:

• The CMIE method significantly improved the performance of seven representative VQA models across multiple benchmarks.
• Demonstrated enhanced generalization capabilities of VQA models when integrated with CMIE.
• Effectively addressed the identified vision-answer bias in VQA tasks.

Conclusions:

• The proposed CMIE method offers an effective solution to vision-answer bias in VQA.
• CMIE enhances the reliability and generalization of various VQA approaches.
• This causality-based approach advances multimodal understanding in AI.