MSB-VQA: Overcoming multiple source biases for robust visual question answering.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Many Visual Question Answering (VQA) models exhibit biases, hindering their ability to reason with multimodal information.
• Existing bias mitigation techniques often focus solely on language bias and yield unsatisfactory results.
• Models excelling on standard VQA datasets falter on bias-sensitive VQA-CP datasets.

Purpose of the Study:

• To propose a novel method, MSB-VQA, that comprehensively targets various sources of bias in VQA models.
• To enhance the robustness and reasoning capabilities of VQA models by addressing multimodal shortcut and distributional biases.
• To achieve significant bias reduction without relying on data balancing or augmentation.

Main Methods:

• Developed a bias detector using generative adversarial networks and knowledge distillation to simulate bias formation and eliminate multimodal shortcut biases.
• Implemented a cosine classifier with adaptive angular margin loss and supervised contrastive loss to combat distributional bias from uneven sample distributions.
• Fused predictions from the cosine classifier and base model to balance performance on in-distribution (ID) and out-of-distribution (OOD) datasets.

Main Results:

• The proposed MSB-VQA method significantly outperforms existing approaches in bias reduction across VQA-CPv2, VQAv2, and VQA-CE datasets.
• Demonstrated effective mitigation of both multimodal shortcut and distributional biases.
• Achieved superior performance without employing data balancing or augmentation techniques.

Conclusions:

• MSB-VQA offers a significant advancement in developing unbiased and robust VQA models.
• The method effectively tackles complex biases, leading to improved performance on diverse VQA benchmarks.
• This approach provides a promising direction for future research in reliable multimodal AI.