FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence

Area of Science:

• Computer Vision
• Machine Learning
• Image Analysis

Background:

• Dense semantic correspondence is crucial for image understanding but challenged by intra-class variations.
• Existing methods struggle with appearance and shape differences within object categories.

Purpose of the Study:

• To develop a robust descriptor for dense semantic correspondence.
• To address intra-class appearance and shape variations in semantic matching.

Main Methods:

• Introduced Fully Convolutional Self-Similarity (FCSS) descriptor.
• Utilized Local Self-Similarity (LSS) via a Convolutional Self-Similarity (CSS) layer.
• Employed a Convolutional Affine Transformer (CAT) layer for shape variation.
• Enabled weakly-supervised learning using object priors and correspondence consistency.

Main Results:

• FCSS demonstrates robustness against intra-class appearance variations.
• The CAT layer effectively handles shape variations among object instances.
• Weakly-supervised learning approach leverages limited training data.
• FCSS significantly outperforms existing handcrafted and CNN-based descriptors.

Conclusions:

• FCSS offers a powerful new approach for dense semantic correspondence.
• The method effectively addresses key challenges in semantic matching.
• FCSS shows superior performance across various benchmarks.