Revisiting Semantic Correspondence: When Feature Aggregation Hurts Structural Integrity

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Semantic correspondence aims to match similar instances across different images.
• Current methods often use features from Stable Diffusion (SD) and DINOv2.
• A key issue is feature aggregation disrupting SD feature structure, harming performance.

Purpose of the Study:

• To address the degradation of semantic matching performance caused by feature aggregation in SD features.
• To propose and validate a novel aggregation method that preserves the structural integrity of SD features.

Main Methods:

• Introduced Structure-Aware Aggregation (SAA) as a replacement for common feature aggregation.
• SAA decomposes SD features into texture and contour components using filtering.
• Aggregates only texture components, preserving contour structures.

Main Results:

• SAA significantly enhances the performance of state-of-the-art semantic correspondence models.
• The method achieves improvements without increasing trainable parameters or computational overhead.
• Demonstrated effectiveness across geometric, cross-species, and cross-family correspondence tasks.

Conclusions:

• Feature aggregation's disruption of SD structural integrity is a critical issue in semantic correspondence.
• SAA effectively preserves essential structural information, boosting performance.
• SAA offers a generalizable and efficient solution for improving semantic correspondence.