MRA-Net: Improving VQA Via Multi-Modal Relation Attention Network.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Visual Question Answering (VQA) aims to answer questions about images.
• Current VQA models often struggle with complex questions due to limited relation modeling and feature integration.

Purpose of the Study:

• To propose a novel end-to-end VQA model, MRA-Net, that addresses limitations in current approaches.
• To improve VQA performance and interpretability by exploring both textual and visual relations.

Main Methods:

• Developed a self-guided word relation attention scheme to capture latent semantic relations between words.
• Introduced two question-adaptive visual relation attention modules for binary and trinary object relations.
• Integrated appearance and relation features for effective multi-modal fusion.

Main Results:

• The MRA-Net model demonstrated superior performance across five benchmark datasets (VQA-1.0, VQA-2.0, COCO-QA, VQA-CP v2, TDIUC).
• The model effectively extracts both fine-grained binary and sophisticated trinary visual relations.
• Improved visual reasoning capabilities were achieved through deeper visual semantics.

Conclusions:

• MRA-Net significantly outperforms state-of-the-art approaches in Visual Question Answering.
• The proposed model offers enhanced capabilities for understanding complex visual and textual information.
• Effective integration of multi-modal features leads to more robust VQA systems.