Physics-Guided Cross-Modal Decoupling with Test-Time Adaptation for Hyperspectral Image Restoration.

Area of Science:

• Remote Sensing
• Computer Vision
• Image Processing

Background:

• Hyperspectral image (HSI) restoration faces challenges due to spectral-spatial coupling and limited data.
• Cross-modal transfer from RGB models shows promise but suffers from high computational costs and spectral distortions.
• Existing methods require fine-tuning and struggle with accurate spectral adaptation.

Purpose of the Study:

• To develop a spectral-faithful HSI restoration method without fine-tuning pretrained RGB models.
• To address computational costs and spectral distortion issues in cross-modal HSI recovery.
• To introduce a novel network architecture for efficient and accurate HSI restoration.

Main Methods:

• Proposed a Two-Stage Cross-Modal Decoupling Network (CMDN) for HSI restoration.
• Utilized Singular Value Decomposition (SVD) to decouple HSIs into spatial and spectral components.
• Developed a Physics-Motivated Spectral Rectifier (PMSR) for unsupervised, sample-specific spectral calibration.

Main Results:

• Achieved spectral-faithful HSI restoration without fine-tuning RGB priors.
• Demonstrated superior spatial reconstruction accuracy and spectral consistency compared to state-of-the-art methods.
• Successfully decoupled spectral-spatial information for improved HSI recovery.

Conclusions:

• CMDN offers an efficient and effective solution for HSI restoration tasks like super-resolution, denoising, and inpainting.
• The proposed method preserves spectral integrity while enhancing spatial details.
• Unsupervised test-time learning with PMSR enables robust spectral calibration.