Latent Diffusion Enhanced Rectangle Transformer for Hyperspectral Image Restoration

Area of Science:

• Remote Sensing
• Computer Vision
• Image Processing

Background:

• Hyperspectral image (HSI) restoration is crucial for various applications.
• Current deep learning methods struggle with HSI's inherent spatial non-local self-similarity and spectral low-rank properties.

Purpose of the Study:

• To develop an advanced deep learning model for HSI restoration.
• To address limitations in capturing spatial and spectral characteristics of HSIs.

Main Methods:

• Proposed a latent diffusion enhanced rectangle Transformer architecture.
• Introduced a multi-shape spatial rectangle self-attention module for non-local spatial similarity.
• Developed a spectral latent diffusion enhancement module for low-rank property extraction using diffusion models.

Main Results:

• Demonstrated superior performance across four HSI restoration tasks: denoising, super-resolution, reconstruction, and inpainting.
• Achieved significant improvements in both objective metrics and subjective visual quality.
• Validated the model's effectiveness on diverse HSI datasets.

Conclusions:

• The proposed method effectively restores HSIs by leveraging spatial non-local similarity and spectral low-rank properties.
• The integration of diffusion models enhances the representation of HSI-specific latent low-rank characteristics.
• This approach offers a promising direction for advanced HSI restoration techniques.