Vision Transformer With Quadrangle Attention.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Window-based attention is popular in vision transformers for efficiency.
• Hand-crafted windows limit adaptability to objects of varying sizes and shapes.

Purpose of the Study:

• To introduce a novel quadrangle attention (QA) method.
• To enhance the flexibility of vision transformers for object recognition.

Main Methods:

• Developed an end-to-end learnable quadrangle regression module.
• Predicted transformation matrices to adapt windows to target quadrangles for attention calculation.
• Integrated QA into vision transformers to create the QFormer architecture.

Main Results:

• QFormer demonstrated superior performance across multiple vision tasks.
• Achieved state-of-the-art results in classification, object detection, semantic segmentation, and pose estimation.
• Showcased minor code modifications and negligible computational overhead.

Conclusions:

• Quadrangle attention offers a flexible and efficient alternative to fixed window-based attention.
• QFormer significantly advances vision transformer capabilities for complex visual recognition.
• The proposed method provides a robust framework for handling diverse object characteristics.