Adaptive 3D Convolution for Remote Sensing Image Fusion

Area of Science:

• Computer Vision
• Remote Sensing
• Deep Learning

Background:

• Remote sensing image fusion combines high-resolution spatial data with low-resolution spectral data.
• Deep learning methods often treat spectral information as feature channels, causing distortions.
• Existing 3D convolutions are computationally expensive and sub-optimal for fusion tasks.

Purpose of the Study:

• To develop a novel deep learning approach for remote sensing image fusion.
• To address spectral distortions and computational inefficiency in current methods.
• To enhance the integration of spatial and spectral information for improved fusion outcomes.

Main Methods:

• Proposed Adaptive 3D Convolution (Ada3D) paradigm for remote sensing image fusion.
• Ada3D generates unique 3D kernels for each input voxel by combining spatial and spectral kernels.
• Incorporated adaptive biases and group convolution for enhanced adaptivity and reduced complexity.

Main Results:

• Ada3D demonstrated state-of-the-art (SOTA) performance across five datasets.
• The method effectively captures fine-grained details and preserves spectral information.
• Achieved full adaptivity in an efficient manner, outperforming existing techniques.

Conclusions:

• Ada3D offers a superior and efficient solution for remote sensing image fusion.
• The adaptive kernel generation effectively integrates multi-source image information.
• The proposed method significantly advances the field of spectral-spatial fusion.