Learning Deep ISP for High-Speed Cameras: Achieving DSLR-Quality Imaging Under High Frame Rates

Area of Science:

• Scientific Imaging
• Computational Photography
• Computer Vision

Background:

• High-speed imaging is crucial for capturing dynamic events but suffers from image degradation (noise, color distortion) due to sensor limitations and extreme acquisition conditions.
• Existing methods struggle to balance high frame rates with acceptable image quality.

Purpose of the Study:

• To develop a deep image signal processing (ISP) paradigm for high-speed cameras.
• To achieve image quality comparable to digital single-lens reflex (DSLR) cameras while maintaining high frame rates.
• To address noise, color distortion, and pixel misalignment in high-speed images.

Main Methods:

• Construction of the RHID dataset, the first large-scale real-world high-speed imaging ISP dataset with RAW and sRGB image pairs.
• Development of a misalignment-robust ISP learning framework (MisISP).
• Implementation of a prior mapper-guided image alignment module (PMIA) and a spectrum-guided weakly-aligned image supervisory loss.

Main Results:

• The proposed deep ISP paradigm significantly improves image quality in high-speed imaging.
• Demonstrated substantial advancements in noise suppression, brightness enhancement, and color preservation.
• Outperformed existing deep ISP models for high-speed imaging applications.

Conclusions:

• The novel deep ISP paradigm effectively enhances image quality for high-speed cameras.
• The RHID dataset and MisISP framework provide valuable resources for future research in high-speed imaging.
• This work bridges the gap between high-speed acquisition capabilities and professional image quality.