Dense Cross-Modal Correspondence Estimation With the Deep Self-Correlation Descriptor.

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Establishing dense correspondences between images is crucial for various computer vision tasks.
• Existing methods often struggle with images from different modalities (e.g., varying spectral ranges or lighting).
• Non-rigid deformations and geometric variations pose significant challenges for accurate image matching.

Purpose of the Study:

• To present a novel descriptor, deep self-correlation (DSC), for robust dense correspondences between cross-modal images.
• To develop a geometry-invariant version (GI-DSC) that addresses scale and rotation variations.
• To offer a training-free, handcrafted alternative to deep learning-based descriptors.

Main Methods:

• DSC encodes local self-similar structures in a pyramidal manner.
• It computes self-correlation surfaces with randomly sampled patches.
• Spatial pyramid pooling in a log-polar configuration encodes feature responses.
• GI-DSC incorporates multi-scale computation and canonical orientation estimation.

Main Results:

• DSC and GI-DSC demonstrate high robustness to non-rigid image deformations.
• The descriptors are effective for cross-modality image matching, handling photometric and geometric variations.
• Extensive experiments show state-of-the-art performance on challenging datasets.
• The proposed methods are training-free and generalize well across different imaging modalities.

Conclusions:

• DSC and GI-DSC are effective handcrafted descriptors for dense cross-modal image correspondence.
• These methods offer significant advantages in robustness and generalization compared to existing approaches.
• The training-free nature makes them readily applicable in various computer vision applications.