Robust Visual Question Answering: Datasets, Methods, and Future Challenges.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Visual Question Answering (VQA) systems face challenges with data biases, leading to poor out-of-distribution performance.
• Existing VQA methods often memorize biases rather than learning grounded image understanding.

Purpose of the Study:

• To provide a comprehensive survey of datasets, evaluation metrics, and debiasing methods for VQA.
• To analyze the robustness of vision-and-language pre-training models in VQA tasks.
• To identify future research directions in robust VQA.

Main Methods:

• Overview of dataset development from in-distribution and out-of-distribution perspectives.
• Examination of evaluation metrics used in VQA datasets.
• Proposal of a typology for existing VQA debiasing methods, analyzing their development, features, and comparisons.
• Analysis of representative vision-and-language pre-training models' robustness on VQA.

Main Results:

• The survey categorizes VQA datasets and evaluation metrics, highlighting their evolution and limitations.
• A structured typology of debiasing methods is presented, detailing their approaches and comparative robustness.
• Analysis reveals the robustness characteristics of current vision-and-language pre-training models in VQA.

Conclusions:

• The study underscores the critical need for robust VQA systems that overcome data biases.
• It synthesizes current research on VQA robustness, offering a foundational resource for researchers.
• Key areas for future research are identified to advance the field of reliable visual question answering.