Pyramidal Semantic Correspondence Networks.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Estimating dense semantic correspondence between images with significant appearance and shape variations is challenging.
• Existing methods often struggle with large intra-class variations and require extensive labeled data.

Purpose of the Study:

• To propose a novel deep architecture, Pyramidal Semantic Correspondence Networks (PSCNet), for accurate estimation of locally-varying affine transformation fields.
• To address limitations in handling appearance and shape variations in semantically similar images.
• To introduce a weakly-supervised training scheme to mitigate insufficient training data.

Main Methods:

• PSCNet employs a pyramidal model for progressive, coarse-to-fine estimation of affine transformation fields, naturally imposing smoothness constraints.
• Two spatial pyramid models are proposed: quad-tree rectangles and semantic element division.
• A novel weakly-supervised training scheme leverages correspondence consistency across image pairs for progressively evolving supervisions.

Main Results:

• The proposed PSCNet method demonstrates superior performance in estimating dense semantic correspondence.
• Experiments on multiple benchmarks (TSS, Proposal Flow, Caltech-101, SPair-71k) validate the effectiveness of the approach.
• The method outperforms state-of-the-art techniques for dense semantic correspondence tasks.

Conclusions:

• PSCNet effectively estimates locally-varying affine transformations for semantically similar images, even with large variations.
• The pyramidal approach and weakly-supervised training scheme offer significant advantages over previous methods.
• The findings advance the field of dense semantic correspondence and image matching.