A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images

Area of Science:

• Computer Vision
• Image Processing
• Computational Imaging

Background:

• Deep neural networks (DNNs) are commonly used for spectral reconstruction (SR), aiming to recover spectra from RGB measurements.
• Current DNNs often incorporate spatial context, but their performance gains over simpler pixel-based methods are marginal.
• Existing methods struggle with context-dependent spectral mapping and robustness to image manipulations.

Purpose of the Study:

• To introduce A++, a novel pixel-based algorithm for spectral reconstruction.
• To enhance spectral reconstruction accuracy and efficiency compared to existing deep learning approaches.
• To demonstrate the robustness and superior performance of A++ in applications like scene relighting.

Main Methods:

• A++ clusters spectra, ensuring neighboring spectra are processed by the same spectral reconstruction (SR) map.
• A polynomial regression framework estimates spectral neighborhoods from RGB values during testing.
• The algorithm utilizes pixel-based processing, avoiding reliance on spatial context for spectral mapping.

Main Results:

• A++ achieves superior spectral reconstruction results compared to leading deep neural networks (DNNs).
• The algorithm demonstrates significantly faster implementation and requires orders of magnitude fewer parameters than DNNs.
• A++ exhibits robustness to image manipulations that alter spatial context, unlike some DNN methods.

Conclusions:

• A++ represents a significant advancement in spectral reconstruction, surpassing current DNN performance.
• The pixel-based approach of A++ offers enhanced robustness and efficiency for spectral recovery.
• In scene relighting, A++ provides superior color accuracy and robustness compared to state-of-the-art DNNs.