Efficient Multi-Task Training with Adaptive Feature Alignment for Universal Image Segmentation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Universal image segmentation models aim for single architecture, multi-task training.
• Current methods use text-based task tokens, which lack inherent task differentiation and cause modality discrepancies.
• Existing alignment methods are computationally expensive and unsuitable for resource-constrained devices.

Purpose of the Study:

• To propose an Adaptive Feature Alignment (AFA) method with a learnable task token for universal image segmentation.
• To address limitations of text-based task tokens and complex modality alignment methods.
• To enhance efficiency and effectiveness in lightweight segmentation models.

Main Methods:

• Introduced a learnable task token that automatically captures inter-task differences from image features and text queries.
• Developed Adaptive Feature Alignment (AFA) by replacing image features with class-specific means for efficient cross-modal alignment.
• Integrated the learnable task token with AFA for unified multi-task training.

Main Results:

• The proposed model with AFA and learnable task token demonstrated superior efficiency and effectiveness compared to baseline models.
• Achieved state-of-the-art performance on ADE20K and Cityscapes datasets with comparable parameter counts.
• Showcased the advantage of learnable task tokens over predefined text-based tokens in capturing task nuances.

Conclusions:

• The AFA method with a learnable task token offers a more effective and efficient solution for universal image segmentation.
• This approach overcomes the limitations of text-based conditioning and complex alignment strategies.
• Enables high-performance segmentation on resource-constrained devices.