Dilated multi-Layer perceptron mixer for faster neural networks.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning Architectures

Background:

• Multi-Layer Perceptron Mixer (MLP-Mixer) shows promise in visual tasks but suffers from quadratic complexity and fixed input sizes.
• Transferring pre-trained MLP-Mixer models to downstream tasks incurs high computational costs and requires weight interpolation.

Purpose of the Study:

• To introduce Dilated MLP-Mixers (DMLP) as a faster backbone for image classification and downstream tasks.
• To address the computational limitations and fixed-size input issues of existing MLP-Mixer methods.

Main Methods:

• Proposed Dilated MLP-Mixers (DMLP) backbone utilizing spatial mixing on non-overlapped and dilated windows.
• Implemented sparse and cross-connections between windows in the window MLP-Mixer block (dilated windows).
• Achieved linear complexity with image resolutions and flexibility for varying input sizes.

Main Results:

• DMLP backbone demonstrates performance comparable to MLP and Transformer models.
• Achieved superior processing speed compared to existing methods.
• Outperformed recent approaches in both accuracy and computational efficiency.

Conclusions:

• DMLP offers a computationally efficient alternative to traditional MLP-Mixer and Transformer models.
• The method shows potential for applications in resource-constrained environments, such as automobile insurance and car damage recognition.