A Pyramid Fusion MLP for Dense Prediction

Area of Science:

• Computer Vision
• Deep Learning Architectures
• Machine Learning

Background:

• MLP-based architectures show promise but struggle with global visual dependencies and multi-scale context.
• Existing methods often lack the ability to capture long-range visual information, hindering performance on dense prediction tasks.

Purpose of the Study:

• To propose a novel MLP-based architecture, Pyramid Fusion MLP (PFMLP), designed to address limitations in capturing global visual dependencies and multi-scale context.
• To enhance the performance of MLP models on dense prediction tasks by incorporating multi-scale feature extraction and fusion.

Main Methods:

• Introduced Pyramid Fusion MLP (PFMLP) architecture with multi-scale pooling and fully connected layers to generate feature pyramids.
• Fused feature pyramids using up-sample layers and additional fully connected layers to integrate multi-scale information.
• Employed diverse down-sample rates to achieve varied receptive fields for capturing both long-range dependencies and fine-grained cues.

Main Results:

• PFMLP achieved comparable results to state-of-the-art CNNs and ViTs on the ImageNet-1K benchmark, establishing itself as a competitive lightweight MLP.
• Under similar computational complexity, PFMLP with larger FLOPs surpassed state-of-the-art CNNs, ViTs, and other MLPs.
• Demonstrated seamless transferability of PFMLP's visual representations to downstream tasks like object detection, instance segmentation, and semantic segmentation, yielding competitive outcomes.

Conclusions:

• PFMLP effectively captures global context and multi-scale information, overcoming limitations of previous MLP-based vision models.
• The proposed architecture offers a competitive and lightweight alternative for various computer vision tasks, including dense prediction.
• PFMLP's strong performance across benchmarks and downstream tasks highlights its potential for advancing vision model development.