Unsupervised Pre-Training With Language-Vision Prompts for Low-Data Instance Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Query-based end-to-end instance segmentation (QEIS) methods, inspired by DETR, excel with large datasets but falter with limited data.
• The performance degradation in low-data regimes is due to the necessity of substantial data for training pivotal queries/kernels that capture localization and shape priors.

Purpose of the Study:

• To develop a novel unsupervised pre-training method to improve QEIS model performance in low-data regimes.
• To address the data dependency of QEIS models by incorporating language-vision prompts.

Main Methods:

• Propose Unsupervised Pre-training with Language-Vision Prompts (UPLVP), a three-part method for pre-training QEIS models.
• Masks Proposal: Generate pseudo masks from unlabeled images using language-vision models.
• Prompt-Kernel Matching & Kernel Supervision: Convert pseudo masks to prompts, match features to kernels, and apply kernel-level supervision for robust learning.

Main Results:

• UPLVP enables QEIS models to converge faster and achieve better performance compared to CNN-based models in low-data settings.
• Experimental validation on MS COCO, Cityscapes, and CTW1500 datasets demonstrates significant performance improvements for QEIS models pre-trained with UPLVP.

Conclusions:

• Unsupervised Pre-training with Language-Vision Prompts (UPLVP) effectively enhances QEIS models in low-data regimes.
• The proposed method overcomes the data limitations of traditional QEIS approaches, offering a viable solution for scenarios with scarce training data.