Learning Semantic Correspondence Exploiting an Object-Level Prior.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Semantic correspondence is crucial for understanding image relationships.
• Existing methods like semantic flow and alignment have limitations in data availability and robustness.
• Object-level priors are needed to improve semantic correspondence accuracy.

Purpose of the Study:

• To develop a robust method for dense semantic correspondence between object instances.
• To leverage foreground masks and synthetic deformations for training a CNN.
• To introduce a novel CNN architecture, SFNet, for improved semantic correspondence.

Main Methods:

• Utilized images with binary foreground masks and synthetic geometric deformations.
• Trained a convolutional neural network (CNN) using masks as a supervisory signal.
• Developed SFNet, a CNN architecture incorporating a differentiable argmax function.
• Employed a combined loss function for mask consistency, flow consistency, and smoothness.

Main Results:

• SFNet effectively establishes dense flow fields between different instances of the same object category.
• The approach provides an object-level prior, balancing semantic flow and alignment methods.
• Achieved state-of-the-art performance on standard semantic correspondence benchmarks.
• Demonstrated the effectiveness of using synthetic deformations and masks for training.

Conclusions:

• The proposed SFNet significantly advances the state of the art in semantic correspondence.
• Leveraging object masks and synthetic deformations offers a practical and effective training strategy.
• SFNet's architecture and loss function contribute to its superior performance and robustness.