Multitask Learning for Visual Question Answering.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Current visual question answering (VQA) models often rely on superficial data correlations, leading to insufficient image grounding.
• This reliance on statistical bias hinders robust understanding and accurate responses in VQA systems.

Purpose of the Study:

• To develop a novel end-to-end architecture for VQA that enhances image grounding.
• To improve multimodality representation learning by integrating VQA with a new image cloze (IC) task.

Main Methods:

• A multitask learning framework combining VQA and an image cloze (IC) task.
• A novel word-masking algorithm, based on part-of-speech, to create the multimodal IC task for better image grounding.
• Shared multimodality representation learning for both VQA and IC tasks.

Main Results:

• The proposed model achieved state-of-the-art, second-best performance on VQA v2.0, VQA-CP v2, and GQA datasets.
• The model demonstrates competitive results with fewer parameters and without requiring additional training data.
• The integration of the IC task effectively promotes more sufficient image grounding.

Conclusions:

• The novel multitask learning approach significantly enhances image grounding in VQA.
• The proposed architecture offers an efficient and effective solution for improving VQA performance.
• This work provides a new direction for developing more robust and grounded visual question answering systems.