Simpler is Better: Feature Guard and Interaction for Semantic Correspondence.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Semantic correspondence links keypoints across different object instances.
• Fusing features from vision foundation models like Stable Diffusion (SD) and DINOv2 improves matching.
• Current fusion methods degrade crucial semantic and structural information in SD features.

Purpose of the Study:

• To address the feature degradation issue in current semantic correspondence methods.
• To propose a novel method, Guard Features (GuFT), to preserve essential feature information.
• To develop a simplified yet effective matching framework, SimBetter.

Main Methods:

• Introduced Guard Features (GuFT) to prevent degradation of semantic and structural information in Stable Diffusion features.
• Developed a Bottleneck-style Non-shared Aggregation and Backward Interaction (NABI) module for efficient feature relationship capture.
• Integrated GuFT and NABI into the SimBetter framework, leveraging insights from vision foundation models.

Main Results:

• SimBetter achieves state-of-the-art performance on benchmark datasets (SPair-71k, AP-10K, PF-PASCAL).
• The framework demonstrates superior performance in geometry-aware, cross-species, cross-family, and cross-dataset tasks.
• SimBetter achieves these results with significantly lower computational cost compared to existing methods.

Conclusions:

• Guard Features (GuFT) effectively prevents feature degradation, enhancing semantic correspondence.
• The SimBetter framework offers a simpler and more efficient approach to semantic correspondence.
• SimBetter shows strong potential for applications like image-video semantic correspondence and sticker editing.