Semi-Dense Feature Matching With Transformers and its Applications in Multiple-View Geometry.

Area of Science:

• Computer Vision
• Machine Learning

Background:

• Traditional local image feature matching relies on sequential detection, description, and matching.
• Dense correspondence methods often struggle in low-texture regions.

Purpose of the Study:

• To propose a novel method for local image feature matching that improves accuracy and robustness.
• To address limitations of existing methods in low-texture areas.

Main Methods:

• A two-level matching strategy: coarse dense matching followed by fine refinement.
• Utilizing self and cross-attention layers within a Transformer architecture for feature descriptor generation.
• Leveraging the global receptive field of Transformers for enhanced feature matching.

Main Results:

• LoFTR significantly outperforms state-of-the-art methods on indoor and outdoor datasets.
• Demonstrated superior performance in the Image Matching Challenge 2021.
• Achieved first rank on two public visual localization benchmarks.

Conclusions:

• The proposed Transformer-based approach effectively handles dense image matching, even in challenging low-texture areas.
• LoFTR shows strong applicability in Structure from Motion (SfM) systems and multiple-view geometry.
• The method represents a significant advancement in local image feature matching and visual localization.