Enhancing Mask Transformer with Auxiliary Convolution Layers for Semantic Segmentation.

Area of Science:

• Computer Vision
• Deep Learning
• Image Segmentation

Background:

• Transformer-based methods like Mask2Former excel in semantic segmentation.
• Mask2Former unifies segmentation tasks but struggles with local features and small objects.

Purpose of the Study:

• To improve Mask2Former's ability to capture local features and segment small objects.
• To enhance semantic segmentation performance without adding inference complexity.

Main Methods:

• Introduced auxiliary convolutional branches to Mask2Former during training.
• These branches capture dense local features on the encoder side.
• Auxiliary layers are removed during inference, ensuring no additional computational cost.

Main Results:

• Achieved state-of-the-art performance on benchmark datasets.
• Demonstrated significant improvements in segmenting small objects and capturing local details.
• Reported 57.6% mIoU on ADE20K and 84.8% mIoU on Cityscapes.

Conclusions:

• The proposed auxiliary branches effectively enhance Mask2Former for semantic segmentation.
• The method improves local feature learning and small object segmentation.
• The approach offers performance gains without compromising inference efficiency.