Multi-Stage Network With Geometric Semantic Attention for Two-View Correspondence Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Geometric Deep Learning

Background:

• Establishing image correspondence is vital for many computer vision tasks.
• High outlier ratios (up to 90%) present significant challenges for existing correspondence methods.
• Current methods struggle to effectively leverage geometric transformation consistency and semantic neighboring information.

Purpose of the Study:

• To develop a robust method for outlier removal in image correspondence, especially under extreme outlier conditions.
• To improve the accuracy of camera pose estimation in challenging scenarios.
• To introduce a novel network architecture that integrates geometric and semantic information.

Main Methods:

• Proposed a Multi-Stage Geometric Semantic Attention (MSGSA) network.
• Developed a multi-branch (MB) module for diverse spatial transformations.
• Incorporated a geometric transformation consistency (GTC) module and a geometric semantic attention (GSA) module using Transformer for efficient information extraction.
• Utilized graph-based representation for geometric semantic information processing.

Main Results:

• MSGSA significantly outperforms state-of-the-art methods in outlier removal.
• Demonstrated superior performance in camera pose estimation, particularly with high outlier prevalence.
• Achieved state-of-the-art results on the YFCC100M and SUN3D datasets.

Conclusions:

• The MSGSA network offers a highly effective solution for image correspondence outlier removal.
• The proposed architecture successfully integrates geometric and semantic information for enhanced performance.
• MSGSA provides a robust approach for challenging computer vision problems with significant noise.