A Self-Supervised Residual Feature Learning Model for Multifocus Image Fusion.

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Multi-focus image fusion (MFIF) aims to create a single, all-focused image from multiple source images.
• A significant challenge in MFIF is the scarcity of labeled multi-focus image datasets for training deep learning models.

Purpose of the Study:

• To propose a novel self-supervised residual feature learning model for multi-focus image fusion.
• To address the data scarcity issue by utilizing image super-resolution as a pretext task.

Main Methods:

• A self-supervised model comprising a feature extraction network and a fusion module.
• Leveraging a newly discovered residual gradient prior for low- and high-resolution (LR-HR) image pairs.
• Utilizing image super-resolution as a pretext task with generated LR-HR pairs and natural images as training data.
• Employing an activity level measurement and boundary refinement within the fusion module to generate decision maps.

Main Results:

• The proposed model successfully extracts residual features from multi-focus images.
• Experimental results, based on both subjective and objective evaluations, show superior performance compared to state-of-the-art fusion algorithms.
• The self-supervised approach demonstrates effectiveness in the absence of extensive labeled multi-focus datasets.

Conclusions:

• The developed self-supervised residual feature learning model offers an effective solution for multi-focus image fusion.
• The integration of image super-resolution as a pretext task and the novel residual gradient prior contribute to the model's success.
• The proposed MFIF method achieves state-of-the-art performance, highlighting its potential for practical applications.