S³-Net: Learning spectral-spatio self-similarity for hyperspectral image super-resolution

As an economically feasible approach for hyperspectral image (HSI) super-resolution, fusing HSI with multispectral image (MSI) utilizes the complementary nature of cross-modality information. Given the common presence of repetitive textures and structures, non-local self-similarity are widely used in natural image super-resolution as an efficient prior information. However, self-similarity in HSI has not been fully explored. Existing methods primarily focus on spatial dimension self-similarity, ignoring the role of cross-band self-similarity and spectral-spatio correlations in HSI, resulting in sub-optimal solutions. In this work, we heuristically propose a new framework (S³-Net) to learn the self-similarity prior across spectral-spatio dimensions comprehensively for reconstructing high-resolution HSI. Specifically, the proposed S³-Net is built upon multiple designed Tribranch Self-Similarity Fusion (TSSF) blocks. Each block features three parallel branches which are responsible for capturing the complex interactions between the spatial and spectral dimensions of the input tensor, and utilizing permutation operations to facilitate this multi-dimensional analysis. We further design and insert a Non-Local Self-Similarity Attention (NLSSA) module for each branch of the TSSF block which efficiently aggregate globally relevant correlation features with minimal computational overhead. Extensive experiments on four HSI datasets, including Houston, Pavia Center, Urban, and Xiongan, demonstrated the superior performance and effectiveness of S³-Net. Code is available at https://github.com/wxy11-27/S3-Net.