TransXNet: Learning Both Global and Local Dynamics With a Dual Dynamic Token Mixer for Visual Recognition

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Integrating convolutions with transformers aims to improve generalization via inductive bias.
• Static convolutions in hybrid networks limit dynamic adaptation and feature fusion with self-attention.
• This leads to suboptimal representation capacity in current CNN-transformer architectures.

Purpose of the Study:

• To address the limitations of static convolutions in hybrid vision networks.
• To propose a novel, lightweight dual dynamic token mixer (D-Mixer) for enhanced feature representation.
• To develop a new hybrid CNN-transformer backbone, TransXNet, for improved performance and efficiency.

Main Methods:

• Introduced a dual dynamic token mixer (D-Mixer) that learns global and local dynamics in an input-dependent manner.
• D-Mixer utilizes an efficient global attention module and an input-dependent depthwise convolution on split feature segments.
• Constructed TransXNet, a hybrid CNN-transformer vision backbone, using D-Mixer as the fundamental building block.

Main Results:

• TransXNet-T achieved 0.3% higher top-1 accuracy than Swin-T on ImageNet-1K with less than half the computational cost.
• TransXNet-S and TransXNet-B demonstrated strong scalability, reaching 83.8% and 84.6% top-1 accuracy, respectively.
• The architecture showed superior generalization on dense prediction tasks compared to state-of-the-art methods at lower computational costs.

Conclusions:

• The proposed D-Mixer effectively overcomes the limitations of static convolutions in hybrid networks.
• TransXNet offers a compelling balance of high accuracy, efficiency, and strong generalization capabilities.
• The D-Mixer approach presents a promising direction for designing efficient and effective vision backbone networks.