SigMa: Semantic Similarity-Guided Semi-Dense Feature Matching

Area of Science:

• Computer Vision
• Machine Learning

Background:

• Current semi-dense feature matching methods often neglect high-level semantic information, focusing primarily on local features.
• This imbalance limits the accuracy and robustness of feature matching in complex scenarios.

Purpose of the Study:

• To introduce SigMa, a novel semantic similarity-guided semi-dense feature matching method.
• To effectively integrate low-level local features and high-level semantic features for improved image matching.

Main Methods:

• A dual-branch feature extractor combining convolutional networks and vision foundation models.
• A cross-domain feature adapter to reconcile feature resolution and dimensionality differences.
• A guided pooling method leveraging semantic similarity for efficient attention computation.

Main Results:

• SigMa achieves a competitive accuracy-efficiency trade-off across various image matching tasks.
• The method demonstrates strong generalization capabilities on diverse datasets.
• Ablation studies confirm the effectiveness of the proposed design components.

Conclusions:

• SigMa offers a robust solution for semi-dense feature matching by synergistically utilizing local and semantic information.
• The guided pooling strategy enhances computational efficiency without significant information loss.
• The proposed method advances the state-of-the-art in detector-free, semi-dense feature matching.