NCNet: Neighbourhood Consensus Networks for Estimating Image Correspondences

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Finding dense correspondences between images is challenging due to appearance variations and repetitive patterns.
• Existing methods often require global geometric models or detailed annotations.

Purpose of the Study:

• To develop a novel, end-to-end trainable convolutional neural network (CNN) for reliable dense image correspondence.
• To enable effective training using weak supervision, reducing the need for manual annotation.
• To demonstrate the model's versatility across different image matching tasks.

Main Methods:

• A CNN architecture analyzing neighborhood consensus patterns in the 4D space of correspondences.
• Utilizing semi-local constraints to disambiguate feature matches.
• Training with weak supervision (matching/non-matching image pairs).

Main Results:

• Achieved state-of-the-art performance on the PF, TSS, InLoc, and HPatches benchmarks.
• Demonstrated effective learning from weak supervision without point-to-point annotations.
• Successfully applied to both category-level and instance-level matching tasks.

Conclusions:

• The proposed neighborhood consensus network effectively addresses challenges in dense image correspondence.
• Weakly supervised training is viable and efficient for correspondence learning.
• The method generalizes well to diverse matching scenarios, setting new performance standards.