Unsupervised Hyperspectral and Multispectral Images Fusion Based on Nonlinear Variational Probabilistic Generative...

Area of Science:

• Remote Sensing
• Computer Vision
• Signal Processing

Background:

• Hardware limitations restrict simultaneous high spatial and spectral resolution in sensor imaging.
• A common trend is fusing low-resolution hyperspectral images (LR-HSI) with high-resolution multispectral images (HR-MSI) for enhanced results.
• Existing fusion methods often rely on linear spectral unmixing, limiting their effectiveness.

Purpose of the Study:

• To propose a novel nonlinear variational probabilistic generative model (NVPGM) for unsupervised fusion of LR-HSI and HR-MSI.
• To address the limitations of linear spectral unmixing in hyperspectral image fusion.
• To develop an efficient and scalable model for generating high-resolution hyperspectral images (HR-HSI).

Main Methods:

• Developed a nonlinear variational probabilistic generative model (NVPGM) based on nonlinear unmixing.
• Modeled the joint likelihood of observed pixels in LR-HSI and HR-MSI using latent abundance vectors.
• Utilized neural networks to realize nonlinear functions for generative conditional distributions, creating a nonlinear spectral mixture model.
• Implemented two neural network-based recognition models for efficient inference of latent representations.
• Employed stochastic gradient variational inference for simultaneous parameter optimization and latent representation inference.

Main Results:

• The proposed NVPGM effectively fuses LR-HSI and HR-MSI in an unsupervised manner.
• The model demonstrates superior performance compared to existing methods on three standard datasets.
• NVPGM achieves real-time processing after unsupervised pre-training on additional data.
• The method successfully retrieves the target HR-HSI via feedforward mapping.

Conclusions:

• NVPGM offers a powerful nonlinear approach for unsupervised hyperspectral image fusion.
• The model overcomes the constraints of linear spectral unmixing, improving fusion accuracy.
• NVPGM presents an efficient and scalable solution for generating high-resolution hyperspectral images from lower-resolution inputs.