Joint Camera Spectral Response Selection and Hyperspectral Image Recovery.

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Hyperspectral image (HSI) recovery from RGB images is challenging due to sensitivity to camera spectral response (CSR).
• Existing methods often require prior knowledge of the exact CSR, limiting their practical application.

Purpose of the Study:

• To develop an efficient convolutional neural network (CNN) for joint camera spectral response (CSR) selection and hyperspectral image (HSI) recovery.
• To improve HSI recovery accuracy and robustness across different camera setups (multi-chip and single-chip).

Main Methods:

• A novel CNN architecture is proposed, integrating a HSI recovery network with a CSR selection layer.
• The method learns spectral nonlinear mapping and spatial similarity for HSI reconstruction.
• A nonnegative sparse constraint is applied for optimal CSR determination during network training.

Main Results:

• The proposed HSI recovery network significantly outperforms state-of-the-art methods in quantitative metrics and visual quality.
• The integrated CSR selection layer consistently identifies the optimal CSR, matching exhaustive search results.
• The method demonstrates strong performance in real-world capture systems and on a new hyperspectral flower dataset.

Conclusions:

• The developed CNN-based approach effectively recovers hyperspectral images from single RGB images by jointly optimizing CSR selection and image reconstruction.
• This method offers a robust and efficient solution for HSI recovery, applicable to various camera configurations and real-world scenarios.
• The approach shows promise for downstream applications like hyperspectral image classification.