Unsupervised Test-Time Adaptation Learning for Effective Hyperspectral Image Super-Resolution With Unknown...

Area of Science:

• Computer Vision
• Remote Sensing
• Machine Learning

Background:

• Hyperspectral image (HSI) super-resolution (SR) often fuses low-resolution HSI with high-resolution (HR) multi-spectral images.
• Accurate SR relies on inferring the latent HR HSI's posterior distribution using image priors and degeneration models.
• Complex imaging environments and unknown degenerations hinder accurate posterior inference.

Purpose of the Study:

• To develop an unsupervised test-time adaptation learning (UTAL) framework for HSI SR that addresses unknown degeneration.
• To improve the accuracy of HSI SR by effectively modeling complex image priors and estimating unknown degenerations.
• To enhance the generalization performance of HSI SR in real-world applications, especially under challenging conditions.

Main Methods:

• A two-stage learning scheme: supervised pre-training of a mutual-guiding fusion module for a content-agnostic prior, followed by unsupervised adaptation using self-guiding and degeneration estimation modules.
• Implicitly learning a shared prior and adapting it to image-specific characteristics for posterior inference.
• Meta-training UTAL on diverse synthetic SR tasks and employing an alternative optimization strategy for faster adaptation and improved generalization.

Main Results:

• The proposed UTAL framework accurately infers the latent HSI posterior by effectively modeling complex priors and estimating unknown degenerations.
• UTAL demonstrates superior generalization performance on HSI SR tasks with various unknown degenerations compared to existing methods.
• The meta-trained UTAL achieves good performance on challenging real-world cases with minimal adaptation steps.

Conclusions:

• The UTAL framework offers a robust solution for HSI SR under unknown degeneration by decoupling prior modeling and adaptation.
• This approach significantly enhances the accuracy and generalization capability of HSI SR in complex and unconstrained imaging scenarios.
• UTAL shows promise for various HSI restoration tasks, outperforming current state-of-the-art methods.