LearnMat: Semantic-Aware Self-Supervision Fine-Grained Visual Recognition

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Self-supervised learning (SSL) shows promise for fine-grained visual recognition (FGVR).
• Existing SSL methods struggle with irrelevant patterns and subtle differences crucial for FGVR.
• Current approaches are predominantly unimodal, neglecting the potential of vision-language models (VLMs).

Purpose of the Study:

• To develop a novel self-supervised learning framework, LearnMat, for enhanced FGVR.
• To address limitations of existing methods in handling irrelevant features and capturing subtle discriminative details.
• To explore the untapped potential of VLMs in self-supervised FGVR.

Main Methods:

• Proposed the LearnMat framework with two key modules: Semantic Awareness Module (SAM) and Insight Extraction Module (IEM).
• SAM utilizes a vision-language-grounded semantic distillation strategy with generic textual attributes for semantic constraints and robustness.
• IEM employs gradient-based signals to highlight subtle differences, localize discriminative regions, and mitigate intra-class variation and inter-class similarity.

Main Results:

• LearnMat effectively filters irrelevant feature interference during training.
• The framework successfully extracts more important and subtle discriminative features.
• Experiments demonstrated significant performance improvements over state-of-the-art methods on multiple FGVR datasets.

Conclusions:

• LearnMat offers a robust and effective approach to self-supervised FGVR.
• The proposed framework enhances fine-grained discrimination by focusing on critical subtle differences.
• LearnMat represents a significant advancement in leveraging VLMs for self-supervised FGVR tasks.