Segment Anything Model Is a Good Teacher for Local Feature Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Local feature detection and description are crucial for computer vision tasks.
• Existing methods often overlook semantic information, limiting performance.
• Traditional semantic segmentation models have limitations in recognizing diverse object classes.

Purpose of the Study:

• To propose SAMFeat, a new approach for local feature learning guided by the Segment Anything Model (SAM).
• To enhance local feature detection and description by integrating category-agnostic semantic information.
• To achieve superior performance in computer vision tasks with limited training data.

Main Methods:

• Utilizing SAM as a teacher model to guide local feature learning.
• Implementing Attention-weighted Semantic Relation Distillation (ASRD) for semantic discrimination.
• Developing Weakly Supervised Contrastive Learning Based on Semantic Grouping (WSC).
• Designing Edge Attention Guidance (EAG) to focus on edge regions.

Main Results:

• SAMFeat demonstrates superior performance in local feature detection and description.
• The method shows significant improvements on tasks like image matching (HPatches) and visual localization (Aachen Day-Night).
• Enhanced semantic understanding leads to better feature representation even with limited training samples.

Conclusions:

• SAMFeat effectively integrates semantic information from SAM to advance local feature learning.
• The proposed methods (ASRD, WSC, EAG) contribute to improved accuracy and robustness.
• SAMFeat represents a significant step forward in data-driven local feature learning for computer vision.