Deep Modular Bilinear Attention Network for Visual Question Answering.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Visual Question Answering (VQA) integrates image understanding and language comprehension.
• Attention mechanisms are crucial in VQA, with dot-product being a common approach.
• Existing VQA models often rely on specific datasets for augmentation.

Purpose of the Study:

• To propose a novel Deep Multimodality Bilinear Attention Network (DMBA-NET) for VQA.
• To improve inter-modality and intra-modality relation modeling using bilinear attention.
• To enhance question understanding through dynamic word vectors and self-attention.

Main Methods:

• Developed DMBA-NET framework utilizing two basic attention units: BAN-GA and BAN-SA.
• Employed Bilinear Attention Network (BAN) for attention calculation.
• Utilized BERT (Bidirectional Encoder Representations from Transformers) for dynamic question encoding and self-attention.

Main Results:

• Achieved 70.85% accuracy on the VQA 2.0 test-std dataset.
• Demonstrated effective inter-modality and intra-modality relation construction.
• Successfully processed visual and language features for accurate answer prediction.

Conclusions:

• DMBA-NET offers a robust framework for VQA tasks.
• The proposed attention units and question encoding methods enhance model performance.
• The model achieves high accuracy without relying on external dataset augmentation.