ReDFeat: Recoupling Detection and Description for Multimodal Feature Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Deep learning for local feature extraction in visible images shows promise but suffers from unstable training.
• Cross-modal scenarios exacerbate training instability due to reliance on pre-training.
• Existing methods often couple detection and description inappropriately, hindering performance.

Purpose of the Study:

• To develop a stable and effective deep learning framework for multimodal local feature learning.
• To improve feature matching and image registration in cross-modal scenarios.
• To enable training from scratch without heavy reliance on pre-training.

Main Methods:

• Introduced a mutual weighting strategy to recouple detection and description constraints.
• Detached weights from backpropagation for stable training and to avoid suppressing indistinct features.
• Proposed the Super Detector with a large receptive field and learnable non-maximum suppression.
• Developed a benchmark for cross-modal image pairs (visible, infrared, near-infrared, SAR).

Main Results:

• ReDFeat features trained with the recoupled strategy surpass state-of-the-art methods.
• The proposed Super Detector improves detection capabilities.
• The model demonstrates stable training from scratch.
• Achieved superior performance in feature matching and image registration tasks.

Conclusions:

• The recoupled detection and description strategy significantly enhances multimodal feature learning.
• ReDFeat offers a stable and effective solution for cross-modal image matching and registration.
• The developed benchmark facilitates future research in multimodal feature learning.