Uni-ISP: Toward Unifying the Learning of ISPs From Multiple Mobile Cameras

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Modern end-to-end image signal processors (ISPs) learn complex mappings between image data formats.
• The proliferation of diverse mobile camera models makes developing individual ISPs for each system unsustainable due to limited versatility.
• Existing learned ISPs lack adaptability across varied camera systems.

Purpose of the Study:

• To introduce Uni-ISP, a unified pipeline for learning image signal processing across diverse mobile cameras.
• To develop a highly accurate and adaptable ISP that overcomes the limitations of traditional, model-specific approaches.
• To enable new applications previously inaccessible to conventional learned ISPs.

Main Methods:

• Leveraging device-aware embeddings within a novel pipeline architecture.
• Learning both forward and inverse image signal processing mappings.
• Implementing a specialized training scheme tailored for diverse camera data.

Main Results:

• Uni-ISP demonstrates significant accuracy improvements in learning forward and inverse ISPs, with gains of +2.4dB/1.5dB PSNR.
• The pipeline shows versatility in enabling novel image processing applications.
• Extensive experiments validate Uni-ISP's adaptability to new, unseen camera models.

Conclusions:

• Uni-ISP offers a unified and adaptable solution for image signal processing across a wide range of mobile cameras.
• The device-aware embedding approach enhances ISP performance and unlocks new application possibilities.
• The developed FiveCam dataset supports further research in multi-camera ISP learning.