Complex Dual-Tree Pyramid Scattering Transformer

Area of Science:

• Computer Vision
• Deep Learning
• Signal Processing

Background:

• Transformer networks are crucial for computer vision but suffer from quadratic computational complexity due to pixel-wise attention.
• Existing methods struggle with computational scaling for high-resolution inputs and multiscale feature representation.

Purpose of the Study:

• To propose a novel Transformer architecture that addresses the computational complexity and enhances feature representation for visual tasks.
• To improve the flexibility, sparsity, and robustness of Transformer networks in multiscale environments.

Main Methods:

• Introduced a complex pyramid scattering Transformer (CPST) incorporating sparse scattering constraints using wavelet basis parameters.
• Utilized a dual-tree complex scattering method to mitigate aliasing and enhance feature robustness.
• Implemented a multihead stepwise pyramid scattering coupling mechanism to enrich directional priors.

Main Results:

• CPST demonstrated superior performance in image classification and video tracking compared to baseline Transformers and other wavelet scattering networks.
• The proposed method effectively slows down the increase in computational complexity with multiresolution inputs.
• Achieved more robust feature representation and increased abundance of directional priors.

Conclusions:

• The dual-tree complex pyramid scattering Transformer offers a reliable and superior solution for visual tasks requiring diverse scale processing.
• CPST effectively balances computational efficiency with enhanced feature representation capabilities.
• The findings suggest a promising direction for developing more efficient and robust Transformer models in computer vision.