Infrared and visible image fusion using GAN with fuzzy logic and Harris Hawks optimization

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Visible and infrared image fusion is crucial for applications like surveillance and medical diagnosis.
• Existing fusion methods have limitations in adaptability and performance.
• Machine learning techniques are increasingly important in image fusion.

Purpose of the Study:

• To propose a novel Fuzzy Generative Adversarial Network (FGAN) integrated with Harris Hawks Optimization (HHO) for visible and infrared image fusion.
• To address limitations of existing methods by dynamically optimizing a Mamdani-type fuzzy logic system.
• To achieve superior adaptability and fusion precision through a hybrid approach.

Main Methods:

• Utilized HHO to tune fuzzy rules and output membership functions, using entropy, PSNR, and SSIM as training targets.
• Employed a Support Vector Machine (SVM) with Frechet Inception Distance (FID) for discriminator training.
• Developed a hybrid framework combining fuzzy logic, HHO, and Generative Adversarial Networks (GANs).

Main Results:

• The proposed FGAN demonstrated remarkable adaptability in infrared and visible image fusion.
• Achieved superior visual quality and fusion precision compared to state-of-the-art techniques.
• Experimental results showed a PSNR up to 55 dB, SSIM up to 0.99, and SF lower than 10 on the TNO dataset.

Conclusions:

• The novel FGAN approach offers a revolutionary paradigm for image fusion.
• The method significantly enhances clarity and information retention in fused images.
• This technique holds great potential for improving remote sensing, surveillance, and medical diagnostics.