P2T: Pyramid Pooling Transformer for Scene Understanding.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Vision transformers achieve state-of-the-art results but face high computational costs due to large image token sequence lengths.
• Existing solutions using single pooling operations to reduce sequence length yield less powerful features.

Purpose of the Study:

• To improve vision transformer performance by addressing the limitations of single pooling operations.
• To adapt pyramid pooling, known for context abstraction, into the backbone network design of vision transformers.

Main Methods:

• Proposed a novel pooling-based Multi-Head Self-Attention (MHSA) mechanism by integrating pyramid pooling.
• Developed a universal vision transformer backbone architecture named Pyramid Pooling Transformer (P2T).

Main Results:

• P2T significantly reduces sequence length while capturing powerful contextual features.
• Experiments show P2T outperforms previous CNN- and transformer-based networks in image classification, semantic segmentation, object detection, and instance segmentation.

Conclusions:

• The Pyramid Pooling Transformer (P2T) offers a superior backbone for various computer vision tasks.
• Integrating pyramid pooling into MHSA is an effective strategy for enhancing vision transformer efficiency and performance.