Better Image Filter for Pansharpening

Area of Science:

• Remote Sensing
• Image Processing
• Computer Vision

Background:

• The modulation transfer function tailored image filter (MTF-TIF) is considered optimal for multispectral image pansharpening due to its ability to simulate camera frequency response and enhance image details.
• However, pre-measured MTFs may not accurately represent acquired panchromatic (PAN) and multispectral (MSI) images, especially after resampling operations like geometric correction or registration.
• Deep learning (DL) methods using MTF-TIF for training data generation may lack generalization consistency between training and testing phases.

Purpose of the Study:

• To investigate the limitations of MTF-TIF in pansharpening and propose alternative, more adaptable image filters.
• To develop novel deep learning frameworks capable of learning optimal image filters that overcome the drawbacks of traditional MTF-TIF.
• To enhance the generalization ability and performance of both traditional and DL-based pansharpening techniques.

Main Methods:

• Proposed a pair of symmetric deep learning frameworks designed to learn optimal image filters.
• Embedded two learnable filters within the frameworks: an anisotropic Gaussian image filter and an arbitrary image filter.
• The frameworks are designed to capture subtle image offsets and maintain the smoothness of the global deformation field.

Main Results:

• The proposed frameworks successfully identified image filters superior to traditional MTF-TIFs.
• The learned filters demonstrated improved pansharpening performance across various satellite datasets.
• The developed methods exhibited stronger generalization capabilities compared to existing approaches.

Conclusions:

• The study confirms that learned image filters derived from deep learning frameworks outperform fixed MTF-TIFs for pansharpening.
• The proposed symmetric frameworks offer a robust solution for learning adaptive image filters, enhancing pansharpening accuracy and generalization.
• These findings suggest a new direction for optimizing image filters in remote sensing and other image fusion applications.