Spatial-frequency domain aggregation upsampling for pan-sharpening

Pan-sharpening, fusing high-resolution panchromatic (PAN) images with low-resolution multispectral (LRMS) to generate high-resolution multispectral (HRMS) images, is critical for enhancing remote sensing image quality. Despite significant advancements in deep learning methods, research on the image upsampling process remains limited. Existing approaches either fail to effectively utilize the information from PAN images or struggle to balance spectral and spatial information, thereby constraining the performance of these models. To alleviate these problems, we propose a novel Spatial-Frequency Domain Aggregation Upsampling (SFAU) method. Our method consists of three core modules: the Dual-Domain Nonlinear Fusion (DDNF), Region-Specific Attention Mechanism (RSAM), and Adaptive Feature Fusion Gate (AFFG). The DDNF module integrates Frequency-Aware Feature Aggregation (FAFA) and Spatial Domain Enhancement techniques, enabling the capture of high-frequency features while refining local structural details. The RSAM module adaptively refines feature representations and preserves spatial-spectral correlations. Finally, the AFFG module effectively combines the outputs from the DDNF and RSAM modules, ensuring a balanced integration of spatial and spectral information. Extensive experiments demonstrate that our method outperforms other popular upsampling techniques and significantly enhances the performance of many leading pan-sharpening models, particularly in high-contrast and spectrally complex regions. Additionally, our approach shows strong generalization in real-world scenarios, highlighting its potential for practical remote sensing applications. Code is available at https://github.com/zacianfans/SFAU.