Learning Raw Image Reconstruction-Aware Deep Image Compressors.

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Conventional image compression (e.g., JPEG) and current deep learning methods prioritize standard RGB (sRGB) fidelity.
• Many computer vision applications require the linear raw-RGB image, which is not the primary output of these compression techniques.
• Prior methods for raw-RGB reconstruction from compressed images were limited by conventional JPEG encoding's lack of awareness for this task.

Purpose of the Study:

• To investigate the capability of deep image compressors to be "aware" of raw-RGB reconstruction as an additional objective.
• To develop a general framework enabling deep networks to jointly optimize for sRGB fidelity and raw-RGB reconstruction errors.
• To evaluate the effectiveness of this joint optimization approach.

Main Methods:

• Developed a general framework for deep image compression networks to incorporate a joint loss function.
• The joint loss function considers both image fidelity errors in the sRGB color space and raw reconstruction errors.
• Two training scenarios were explored: training from scratch with the joint loss and fine-tuning pre-trained networks.

Main Results:

• The proposed joint loss function significantly improves the Peak Signal-to-Noise Ratio (PSNR) of raw reconstructions.
• This improvement in raw reconstruction is achieved with only a minor impact on sRGB fidelity, as measured by Multi-Scale Structural Similarity Index Measure (MS-SSIM).
• Both training from scratch and fine-tuning approaches demonstrated the benefits of the joint loss.

Conclusions:

• Deep learning-based image compressors can be effectively trained to be aware of raw-RGB reconstruction objectives.
• Jointly optimizing for sRGB fidelity and raw-RGB reconstruction offers a superior approach compared to sRGB fidelity-only compression.
• This framework enhances the utility of deep image compression for computer vision tasks requiring linear raw image data.