A General Image Fusion Approach Exploiting Gradient Transfer Learning and Fusion Rule Unfolding.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Existing deep learning image fusion methods lack efficiency in model training and network design.
• Current approaches fail to effectively utilize complementary information across diverse fusion tasks.
• Heuristic-based network designs limit the versatility of general image fusion models.

Purpose of the Study:

• To propose a comprehensive deep learning framework for general image fusion.
• To address limitations in model training and network design for single-model multi-task fusion.
• To develop a versatile and efficient image fusion network for practical applications.

Main Methods:

• Developed a sequential gradient-transfer framework to leverage complementary information across tasks.
• Proposed fusion rule unfolding integrated into a deep equilibrium model for network design.
• Utilized gradient transfer learning for enhanced information extraction during training.

Main Results:

• The proposed method achieves superior image fusion results across multi-focus, multi-exposure, and infrared/visible tasks.
• Generated images exhibit richer structural information and competitive objective metrics.
• Demonstrated significant performance improvements on unseen medical image fusion tasks.

Conclusions:

• The novel framework offers an efficient and versatile solution for general image fusion.
• Gradient transfer learning and fusion rule unfolding enable effective multi-task learning.
• The method shows strong generalization capabilities for various image fusion applications.