Research on visual question answering based on dynamic memory network model of multiple attention mechanisms

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Existing visual question answering (VQA) models struggle with complex questions due to a lack of long-term memory, leading to information loss.
• Effective VQA requires robust mechanisms for continuous information processing and contextual understanding.

Purpose of the Study:

• To propose a novel dynamic memory network model (DMN-MA) that integrates a multiple attention mechanism for improved VQA accuracy.
• To address the limitations of current VQA models in handling complex questions and retaining effective information.

Main Methods:

• The study introduces a dynamic memory network model (DMN-MA) incorporating a multiple attention mechanism (combining channel and spatial attention).
• The multiple attention mechanism is applied within the situational memory module for relevant visual vector acquisition.
• The model employs continuous memory updating, storage, and iterative inference, utilizing contextual information for answer generation.

Main Results:

• The proposed DMN-MA model achieved an accuracy of 64.57% on the COCO-QA dataset.
• The model reached an accuracy of 67.18% on the VQA2.0 dataset.
• These results demonstrate significant improvements in VQA performance.

Conclusions:

• The DMN-MA model effectively addresses information loss in VQA by incorporating a multiple attention mechanism into memory networks.
• The integration of channel and spatial attention enhances the model's ability to utilize contextual information for accurate question answering.
• The proposed model represents a significant advancement in the field of visual question answering.