Robust visual question answering via polarity enhancement and contrast

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Visual Question Answering (VQA) models often rely on question-answer correlations, neglecting visual content.
• This reliance on language priors weakens the model's ability to truly understand image-text relationships.

Purpose of the Study:

• To propose an unbiased VQA method that mitigates language priors.
• To enhance the correlation between visual content and textual information in VQA models.

Main Methods:

• A novel two-module model architecture was designed.
• The Answer Visual Attention Modules generate positive predictions, while the Dual Channels Joint Module generates negative predictions.
• A new loss function was developed to train the model using positive and negative predictions alongside the correct answer.

Main Results:

• The proposed method achieved 61.24% performance on the VQA-CP v2 dataset.
• Unlike existing debiasing methods, this approach improved performance on both VQA v2 and VQA-CP v2 datasets without performance degradation on the VQA v2 dataset.

Conclusions:

• The developed unbiased VQA method effectively addresses language priors.
• The model demonstrates improved performance across multiple VQA benchmarks, highlighting its robustness and generalizability.