Stitching Misaligned Multi-Spectral Images by Versatile Transformation: A Streamlined Solution and Applications

Area of Science:

• Computer Vision
• Image Processing
• Remote Sensing

Background:

• Multi-spectral image stitching requires accurate registration of image pairs from different viewpoints.
• Stitching accuracy significantly decreases with misaligned multi-modality (e.g., infrared and visible) images.
• Current methods often process multi-modality registration and multi-view alignment independently, limiting robustness.

Purpose of the Study:

• To develop a robust multi-spectral image stitching method for misaligned infrared-visible image pairs.
• To concurrently address multi-modality registration and multi-view alignment challenges.
• To improve the accuracy and robustness of multi-spectral panorama reconstruction.

Main Methods:

• A hierarchical versatile transformation for progressive correspondence matching, combining sparse homography for global adjustment and point-flexible splines for local fine-tuning.
• A dual consistency-driven modality transfer to mitigate feature variance and facilitate registration between infrared and visible images.
• Encoding multi-spectral image pairs into a latent domain for adaptive exploitation of complementary information during cross-view alignment.

Main Results:

• The proposed method achieves robust reconstruction of multi-spectral panoramas by leveraging consistency between multi-modality and multi-view matching.
• Experimental results demonstrate the superiority of the method in both registration and stitching tasks compared to existing approaches.
• The concurrent processing of registration and alignment enhances overall stitching accuracy and reliability.

Conclusions:

• The integrated approach of concurrent multi-modality registration and multi-view alignment offers a significant improvement for multi-spectral image stitching.
• The developed hierarchical transformation and dual consistency transfer effectively handle feature variations and misalignments.
• This work provides a more robust solution for reconstructing comprehensive scenes from diverse spectral image data.