Universal Representation for Real-World Misaligned Infrared-Visible Image Fusion

Area of Science:

• Computer Vision
• Image Processing
• Artificial Intelligence

Background:

• Deep learning methods for infrared and visible image fusion often struggle with pre-aligned inputs or implicit alignment, leading to registration errors and loss of semantic structure.
• Existing approaches fail to fundamentally address the amplification of registration errors and the loss of semantic structure in fused results.

Purpose of the Study:

• To propose a universal representation and end-to-end framework (URMIF) for jointly registering and fusing unaligned infrared-visible image pairs.
• To address limitations in current deep learning fusion methods by improving alignment robustness and preserving semantic integrity.

Main Methods:

• URMIF maps images into modality-invariant and modality-specific features for stable alignment and complementary fusion.
• A bi-directionally coupled registration-fusion module performs hierarchical deformation estimation with feedback regularization.
• A dominant-plane prior is introduced as a scene-level constraint for geometric consistency and semantic reliability.

Main Results:

• Extensive experiments on a large-scale dataset (1,500+ pairs) demonstrate robust alignment and high-quality fusion of misaligned inputs.
• The framework significantly reduces artifacts and improves performance in downstream tasks like detection and segmentation.
• URMIF achieves state-of-the-art results on various benchmarks.

Conclusions:

• The proposed URMIF framework effectively achieves joint registration and fusion of unaligned infrared-visible images.
• This approach enhances visual perception robustness across diverse conditions by mitigating registration errors and preserving semantic structure.
• URMIF offers a significant advancement in image fusion technology, with practical implications for various computer vision applications.