Learning to Recover Spectral Reflectance From RGB Images.

Area of Science:

• Computer Vision
• Computational Imaging
• Color Science

Background:

• Spectral reflectance recovery (SRR) from RGB images is crucial for various applications.
• Existing methods often rely on synthetic data and fixed parameters, leading to suboptimal performance on real-world images.
• Capturing ground-truth spectral reflectance and camera spectral sensitivity is data-intensive and costly.

Purpose of the Study:

• To develop a novel approach for accurate spectral reflectance recovery from RGB images.
• To address the limitations of existing methods by incorporating image-specific information.
• To adapt the self-supervised meta-auxiliary learning (MAXL) strategy for SRR.

Main Methods:

• A self-supervised meta-auxiliary learning (MAXL) strategy is employed to fine-tune network parameters using individual testing images.
• A novel network architecture is proposed, integrating physical relationships between spectral reflectance and RGB images based on mathematical analysis.
• Spectral reflectance is recovered from RGB images captured under multiple illuminations to reduce unknowns.

Main Results:

• The proposed network and MAXL strategy demonstrate significant effectiveness in spectral reflectance recovery.
• Qualitative and quantitative evaluations confirm the superiority of the developed approach.
• The method successfully combines external knowledge with internal image information for improved SRR.

Conclusions:

• The proposed MAXL strategy and novel network architecture offer a significant advancement in spectral reflectance recovery.
• The approach effectively leverages image-specific information for enhanced performance on real-world data.
• This work provides a valuable contribution to the field of computational imaging and color science.