Semantic Concentration for Self-Supervised Dense Representations Learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Image-level self-supervised learning (SSL) has advanced significantly, but learning dense patch representations remains difficult.
• Existing methods suffer from patch over-dispersion, where patches from the same instance scatter, degrading performance on dense tasks.
• Image-level SSL implicitly addresses over-dispersion through semantic concentration, which is not directly applicable to dense SSL.

Purpose of the Study:

• To develop explicit semantic concentration methods for dense self-supervised learning (SSL).
• To address the challenges of spatial sensitivity and complex scenes in dense SSL.
• To improve the learning of dense representations for patches in self-supervised learning.

Main Methods:

• Proposed patch correspondence distillation to break strict spatial alignment and learn patch relationships.
• Introduced a noise-tolerant ranking loss, extending Average Precision (AP) loss for robust pseudo-label learning.
• Developed an object-aware filter using cross-attention to represent patches via learnable object prototypes, enabling object-based space mapping.

Main Results:

• The proposed methods effectively mitigate the over-dispersion phenomenon in dense SSL.
• Patch correspondence distillation and the noise-tolerant ranking loss improve the learning from noisy pseudo-labels.
• The object-aware filter successfully discriminates shared patterns in complex scenes by creating an object-based representation space.

Conclusions:

• Explicit semantic concentration is a viable and effective approach for dense self-supervised learning.
• The developed techniques significantly enhance the quality of dense representations learned via SSL.
• Empirical results across various tasks validate the proposed method's effectiveness in improving downstream performance.