Transformer Module Networks for Systematic Generalization in Visual Question Answering.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Transformers demonstrate strong performance in Visual Question Answering (VQA).
• The systematic generalization capabilities of Transformers in VQA, particularly with novel concept combinations, remain unclear.
• Neural Module Networks (NMNs) show promising systematic generalization, even with CNN-based modules.

Purpose of the Study:

• To investigate the benefits of modularity in enhancing Transformer performance for VQA.
• To introduce a novel architecture, Transformer Module Network (TMN), that integrates modularity into Transformers.
• To evaluate TMN's systematic generalization capabilities compared to existing methods.

Main Methods:

• Developed Transformer Module Network (TMN), a novel NMN architecture.
• TMN utilizes compositions of Transformer modules for VQA.
• Evaluated TMN on three VQA datasets, focusing on systematic generalization performance.

Main Results:

• TMNs achieve state-of-the-art systematic generalization performance on three VQA datasets.
• TMNs demonstrate over 30% improvement compared to standard Transformers on novel sub-task compositions.
• Both module composition and module specialization contribute significantly to performance gains.

Conclusions:

• Modularity can significantly enhance the systematic generalization of Transformers in VQA.
• Transformer Module Networks (TMNs) offer a promising direction for improving VQA systems.
• Module specialization within TMNs is crucial for achieving superior performance on novel VQA challenges.