General cross-modality registration framework for visible and infrared UAV target image registration

Area of Science:

• Computer Vision
• Remote Sensing
• Artificial Intelligence

Background:

• Multi-modality image pairs offer complementary information crucial for image-guided Unmanned Aerial Vehicle (UAV) target detection.
• Real-world scenarios frequently present misaligned multi-modality images, posing significant challenges for image registration due to spatial deformation and cross-modality discrepancies.

Purpose of the Study:

• To develop a robust framework for cross-modality image registration, specifically addressing the misalignment of visible and infrared images for UAV applications.
• To simplify complex cross-modality registration into a more manageable mono-modality registration task.

Main Methods:

• Introduction of the General Cross-Modality Registration (GCMR) Framework, utilizing a generation-registration pattern.
• Implementation of an Image Cross-Modality Translation Network (ICMTN) to generate pseudo infrared images from visible images, correcting structural distortions.
• Employment of a Multi-level Residual Dense Registration Network (MRDRN) for enhanced feature extraction and mutual information exploitation in mono-modality registration.

Main Results:

• The proposed GCMR framework, incorporating ICMTN and MRDRN, demonstrates superior performance in registering misaligned visible and infrared images.
• Extensive experiments on public Anti-UAV datasets show state-of-the-art results across five evaluated variants of the architecture.
• The approach effectively overcomes challenges related to spatial deformation and cross-modality discrepancies in image registration.

Conclusions:

• The developed GCMR framework provides an effective solution for visible-infrared image registration in UAV target detection tasks.
• The novel combination of ICMTN and MRDRN significantly improves registration accuracy and robustness.
• This work advances the field of image registration for all-day-all-weather surveillance and reconnaissance applications.