Deep Selective Fusion of Visible and Near-Infrared Images Using Unsupervised U-Net.

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Visible (VIS) images suffer from low contrast and noise in low light.
• Near-infrared (NIR) images offer clear textures but lack color and can have flash artifacts.
• Multispectral fusion aims to combine VIS and NIR advantages for high-quality images.

Purpose of the Study:

• To propose a deep selective fusion method for VIS and NIR images.
• To overcome limitations of existing fusion methods, such as low contrast in VIS and flash effects in NIR.
• To achieve high-quality image fusion using an unsupervised U-Net architecture.

Main Methods:

• Utilized an unsupervised U-Net for deep selective fusion of VIS and NIR images.
• Formulated an energy function as a loss function for unsupervised learning due to lack of ground truth.
• Employed data augmentation and synthesized training data by degrading VIS and masking NIR images.
• Extracted VIS features using a pre-trained VGG network and NIR edge information via an encoding network.

Main Results:

• The proposed fusion network generated visually pleasing images with fine details and natural colors.
• The method effectively reduced noise and mitigated issues like low contrast and flash-like effects.
• Experimental results showed superiority over state-of-the-art methods in both visual quality and quantitative metrics.

Conclusions:

• Deep selective fusion using unsupervised U-Net is effective for combining VIS and NIR images.
• The approach successfully addresses challenges in low-light image fusion.
• The method offers a promising solution for generating high-fidelity multispectral images.