Two-View Correspondence Learning With Local Consensus Transformer.

Area of Science:

• Computer Vision
• Multiview Geometry
• Machine Learning

Background:

• Feature matching is essential in computer vision but challenged by outliers (mismatches).
• Traditional methods struggle with high outlier ratios, impacting multiview geometry applications.
• Local consensus (LC) offers a promising approach to robust feature matching.

Purpose of the Study:

• To introduce a novel trainable neural network, the LC transformer (LCT), for robust feature matching.
• To leverage local consensus (LC) principles within a deep learning framework.
• To enhance performance in wide-baseline stereo vision tasks, particularly with significant outliers.

Main Methods:

• Developed a dynamic graph-based embedding layer to establish neighbor topology.
• Utilized multihead self-attention with channel attention (CA) guided by local topologies.
• Incorporated order-aware graph pooling for global context extraction.
• Designed the LC transformer (LCT) architecture tailored for wide-baseline stereo.

Main Results:

• The LC transformer (LCT) demonstrates significant benefits from incorporating local consensus (LC).
• Achieved state-of-the-art performance on challenging YFCC100M outdoor and SUN3D indoor datasets.
• The model successfully handles scenes with over 90% outliers, showcasing high robustness.

Conclusions:

• Local consensus (LC) is a valuable concept for improving deep learning-based feature matching.
• The LC transformer (LCT) provides a robust and effective solution for wide-baseline stereo matching.
• This approach advances the field of computer vision by addressing challenging outlier scenarios.