RCVQA: Visual question answering model based on reading comprehension.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Current Visual Question Answering (VQA) models struggle with deep semantic analysis due to limited interdisciplinary interaction.
• Existing VQA systems often provide limited, single-word answers, failing to meet diverse user needs.
• Over-reliance on accuracy as an evaluation metric hinders comprehensive performance assessment and model optimization.

Purpose of the Study:

• To introduce RCVQA, a novel reading-comprehension-based VQA model addressing current limitations.
• To develop innovative methods for multi-span answer retrieval, including answer position and content prediction.
• To propose new evaluation metrics for a more thorough assessment of VQA system performance.

Main Methods:

• Preprocessing dataset text to remove irrelevant information for enhanced contextual focus.
• Developing RCVQA model variants (RCVQA_P, RCVQA_C, VQA_T) and testing across four datasets.
• Implementing answer position and content prediction algorithms for multi-span answer retrieval.
• Introducing four novel evaluation metrics (PPR_∥, TPR_∥, H_∥-Means, ESM_∥) for comprehensive VQA system evaluation.
• Integrating image captioning and local training strategies for improved content understanding and data security.

Main Results:

• RCVQA significantly outperforms state-of-the-art methods on multiple benchmarks.
• Achieved improvements ranging from 1% to 7% across datasets like A-OKVQA, KR-VQA, GQA, and OK-VQA.
• Demonstrated enhanced performance in tasks requiring deeper semantic understanding and more comprehensive answers.

Conclusions:

• RCVQA represents a significant advancement in Visual Question Answering.
• The proposed methodology and evaluation metrics offer a more robust approach to VQA.
• The RCVQA model provides more diverse, complete, and accurate answers, paving the way for broader practical applications.