DisAVR: Disentangled Adaptive Visual Reasoning Network for Diagram Question Answering

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Diagram Question Answering (DQA) requires both diagram understanding and reasoning.
• Visual semantic ambiguity in diagrams poses challenges for representation and understanding.
• Content-rich diagrams necessitate flexible and adaptive reasoning for diverse questions.

Purpose of the Study:

• To propose a novel network, Disentangled Adaptive Visual Reasoning Network (DisAVR), for Diagram Question Answering (DQA).
• To jointly optimize the dual processes of representation and reasoning in DQA.
• To address visual semantic ambiguity and enable adaptive reasoning on diagrams.

Main Methods:

• Developed an improved region feature learning module integrating patch, text, and region features.
• Implemented a question parsing module to guide reasoning with region, spatial, and semantic information.
• Designed a disentangled adaptive reasoning module with visual reasoning cells and an adaptive routing mechanism.

Main Results:

• The proposed DisAVR network effectively handles visual semantic ambiguity in diagrams.
• The adaptive routing mechanism allows for flexible exploration of optimal reasoning paths.
• Experiments on three DQA datasets demonstrate the superiority of the DisAVR model.

Conclusions:

• Disentangled Adaptive Visual Reasoning Network (DisAVR) offers a superior approach to Diagram Question Answering (DQA).
• The model's ability to jointly optimize representation and reasoning enhances performance on complex diagrams.
• DisAVR effectively addresses challenges related to visual semantic ambiguity and adaptive reasoning.