NExT-OOD: Overcoming Dual Multiple-Choice VQA Biases.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Multiple-choice Visual Question Answering (VQA) models show progress but rely on dataset correlations, limiting multimodal understanding and generalization.
• Spurious correlations, specifically Vision-Answer (VA) bias and Question-Answer (QA) bias, hinder VQA model performance.

Purpose of the Study:

• To systematically investigate VA and QA biases in VQA models.
• To develop a benchmark for evaluating VQA model generalizability and reasoning in out-of-distribution settings.
• To propose a novel method for reducing identified biases in VQA models.

Main Methods:

• Construction of the NExT-OOD benchmark with three sub-datasets (NExT-OOD-VA, NExT-OOD-QA, NExT-OOD-VQA) for bias quantification.
• Evaluation of existing VQA models on NExT-OOD to demonstrate performance degradation.
• Proposal of a graph-based cross-sample method with a contrastive graph matching loss for bias reduction.

Main Results:

• Existing VQA models exhibit significant performance degradation on the NExT-OOD benchmark compared to standard datasets.
• The proposed graph-based method effectively mitigates VA and QA biases by leveraging cross-sample information.
• The approach encourages models to focus on multimodal content rather than spurious correlations.

Conclusions:

• The NExT-OOD benchmark provides a scientific tool for assessing VQA model generalizability and reasoning.
• The proposed bias reduction method significantly outperforms existing strategies.
• The developed approach demonstrates effectiveness and generalizability for improving VQA models.