A Multitask Network for Joint Multispectral Pansharpening on Diverse Satellite Data

Area of Science:

• Remote Sensing
• Computer Vision
• Machine Learning

Background:

• Convolutional Neural Network (CNN)-based multispectral (MS) pansharpening methods typically require separate training for each satellite dataset.
• This single-task learning (STL) approach overlooks potential correlations between different satellite image datasets.

Purpose of the Study:

• To develop a unified framework for joint MS pansharpening across diverse satellite imagery.
• To address the limitations of STL by proposing a multitask learning (MTL) paradigm.

Main Methods:

• A multitask network (MTNet) is proposed, treating pansharpening for each satellite as a distinct task within a unified framework.
• MTNet employs a task-agnostic subnetwork (TASNet) for knowledge sharing and task-specific subnetworks (TSSNets) for adaptation.
• TASNet integrates Transformer modules to capture global information, overcoming CNNs' local connectivity limitations.
• Band-aware dynamic convolutions (BDConvs) are introduced to adapt receptive field sizes for varied ground scenes and spectral bands.

Main Results:

• Experimental results across multiple datasets demonstrate the superiority of the proposed MTNet.
• The approach effectively shares generic knowledge while adapting to specific satellite data characteristics.
• The integration of Transformer modules and BDConvs enhances the model's ability to process diverse remote sensing data.

Conclusions:

• The proposed MTNet offers a more efficient and effective solution for MS pansharpening compared to existing STL methods.
• MTL provides a robust framework for leveraging inter-dataset correlations in remote sensing image processing.
• The novel components, TASNet with Transformers and BDConvs, significantly contribute to improved pansharpening performance.