Exploring Sparse Spatial Relation in Graph Inference for Text-Based VQA.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Text-based Visual Question Answering (TextVQA) involves complex relational inference between numerous objects and Optical Character Recognition (OCR) tokens.
• Existing methods often process all visual relationships, leading to redundancy and inefficiency.
• Identifying and utilizing only the most pertinent relationships is crucial for improving TextVQA performance.

Purpose of the Study:

• To address the challenge of redundant relational inference in TextVQA.
• To develop a novel method for identifying and pruning superfluous visual connections.
• To enhance the accuracy and interpretability of TextVQA models.

Main Methods:

• Propose a Sparse Spatial Graph Network (SSGN) incorporating a spatially aware relation pruning technique.
• Utilize spatial factors like distance, geometric dimension, overlap area, and DIoU for pruning.
• Employ a progressive graph learning architecture considering object-object, OCR-OCR, and object-OCR token relationships.

Main Results:

• SSGN demonstrates promising performance on TextVQA and ST-VQA datasets.
• The proposed spatially aware pruning effectively reduces redundant relational inference.
• Visualization results confirm the interpretability of the SSGN method.

Conclusions:

• The SSGN effectively prunes redundant relationships in TextVQA, leading to improved performance.
• Spatially aware relation pruning is a viable technique for enhancing visual reasoning in complex scenes.
• The SSGN model offers an interpretable approach to TextVQA by focusing on pivotal visual connections.