ALSA: Adversarial Learning of Supervised Attentions for Visual Question Answering.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Visual Question Answering (VQA) relies on attention mechanisms to link questions with relevant image regions.
• Existing VQA methods struggle with attention by either using limited region types or neglecting human attention priors.
• This limitation hinders accurate inference for questions involving foreground objects and background context.

Purpose of the Study:

• To propose a novel VQA model, Adversarial Learning of Supervised Attentions (ALSAs), that addresses limitations in current attention mechanisms.
• To incorporate prior knowledge of human attention into the VQA attention distribution learning process.
• To improve the VQA model's ability to focus on question-relevant image regions for more accurate answer inference.

Main Methods:

• Developed two supervised attention modules: free-form based and detection-based, to leverage prior knowledge.
• Implemented an adversarial learning mechanism for interplay between the two attention modules.
• Utilized adversarial learning to mutually reinforce modules, enhancing multi-view feature correlation for answer inference.

Main Results:

• The proposed ALSAs model demonstrated favorable performance across three common VQA datasets.
• The integration of supervised attention modules and adversarial learning improved focus on question-relevant image areas.
• The method effectively learned correlations between questions and images from both free-form and detection-based views.

Conclusions:

• ALSAs offers a significant advancement in VQA by effectively utilizing human attention priors.
• The adversarial learning framework enhances the robustness and accuracy of attention mechanisms in VQA.
• The model's superior performance validates the approach for complex visual question answering tasks.