A Real-Time FPGA Implementation of Infrared and Visible Image Fusion Using Guided Filter and Saliency Detection

Area of Science:

• Computer Vision
• Image Processing
• Hardware Acceleration

Background:

• Infrared and visible light image fusion offers enhanced visual perception for applications like autonomous driving and surveillance.
• Existing fusion methods struggle with satisfactory speed and quality, particularly for real-time applications.
• The need for improved fusion techniques is critical for advancing technologies reliant on multi-sensor data.

Purpose of the Study:

• To propose a multi-scale FPGA-based image fusion technology that enhances both visual capability and fusion speed.
• To address the limitations of current image fusion methods in terms of processing speed and output quality.
• To develop a real-time image fusion system suitable for demanding environments and applications.

Main Methods:

• Image decomposition into detail, saliency, and background layers using guided filter and saliency detection.
• Attention mechanism-based fusion weight map generation for the saliency layer.
• Multi-scale fusion strategies (weight fusion for saliency/detail, average for background) combined with contrast enhancement.
• Hardware circuit design using high-level synthesis for FPGA implementation (XCZU15EG board).

Main Results:

• The proposed method achieves fast real-time image fusion at 55 frames per second for 640x470 resolution infrared and visible images.
• Demonstrated effective image enhancement in challenging conditions such as glare and smoke.
• Substantially improved visual enhancement capability and fusion speed compared to existing methods.
• Successful hardware implementation and testing on the XCZU15EG FPGA board.

Conclusions:

• The multi-scale FPGA-based image fusion technology provides a significant advancement in real-time visual enhancement.
• The method effectively addresses the speed and quality limitations of previous image fusion techniques.
• This technology holds great promise for improving performance in critical applications like autonomous driving and military reconnaissance.