Generalized Semantic Contrastive Learning via Embedding Side Information for Few-Shot Object Detection.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Few-shot object detection (FSOD) aims to identify novel objects using minimal training data.
• Existing fine-tuning methods struggle with feature generalization and overfitting due to limited samples for novel categories.
• Challenges include inseparable classifier boundaries and insufficient data representation for novel categories.

Purpose of the Study:

• To develop a generalized feature representation learning method for FSOD that addresses limitations of existing approaches.
• To improve the model's ability to adapt to unknown scenarios by constructing a generalized feature space.
• To alleviate negative influences from feature space and sample viewpoints using side information.

Main Methods:

• Utilized embedding side information to create a knowledge matrix quantifying semantic relationships between base and novel categories.
• Developed contextual semantic supervised contrastive learning with embedded side information to enhance discrimination between similar categories.
• Introduced a side-information guided region-aware masked module to augment sample diversity and prevent overfitting from sparse samples.

Main Results:

• The proposed model theoretically reduces the upper bound of generalization error.
• Extensive experiments on PASCAL VOC, MS COCO, LVIS V1, FSOD-1K, and FSVOD-500 benchmarks show superior performance.
• Significant improvements in FSOD capabilities across most shots/splits were demonstrated using ResNet and ViT backbones.

Conclusions:

• The novel generalized feature representation learning method effectively addresses key challenges in few-shot object detection.
• The integration of side information and advanced learning techniques leads to enhanced feature discrimination and generalization.
• The proposed approach represents a significant advancement in few-shot object detection, offering improved accuracy and robustness.