Improved mapping between illuminations and sensors for RAW images.

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• RAW images contain sensor-specific RGB values and color casts due to illumination.
• Capturing RAW datasets for deep learning is challenging due to sensor and illumination variations.
• Illumination augmentation and cross-sensor mapping are crucial for reducing data capture burdens.

Purpose of the Study:

• To address the challenges of capturing RAW image datasets for deep learning.
• To introduce a novel dataset for illumination and sensor mapping.
• To develop an efficient method for RAW image mapping across different sensors and illuminations.

Main Methods:

• A customized lightbox with tunable illumination spectra was used to capture scenes with multiple cameras.
• A new dataset comprising 390 illuminations, four cameras, and 18 scenes was created.
• A lightweight neural network was developed for illumination and sensor mapping.

Main Results:

• The proposed lightweight neural network approach for illumination and sensor mapping demonstrated superior performance compared to existing methods.
• The dataset facilitates research in RAW image processing and cross-sensor calibration.
• The utility of the approach was validated on the downstream task of training a neural Image Signal Processor (ISP).

Conclusions:

• The developed dataset and neural network approach effectively address the challenges of RAW image variations.
• This work significantly reduces the burden of data capture for deep learning in computational photography.
• The findings pave the way for more robust and adaptable deep learning models in image processing.