High dynamic range imaging based on an asymmetric event-SVE camera system.

Area of Science:

• Computer Vision and Image Processing
• Computational Imaging
• Sensor Fusion

Background:

• Conventional cameras struggle with high dynamic range (HDR) imaging due to overexposure in extreme illumination.
• Event cameras offer high temporal resolution and HDR, while spatially varying exposure (SVE) sensors provide single-shot radiometric diversity.

Purpose of the Study:

• To develop a hardware-algorithm co-designed HDR imaging system integrating SVE and event sensors.
• To address challenges in non-coaxial geometry and heterogeneous optics for effective cross-modal alignment.
• To improve highlight recovery, edge fidelity, and robustness in HDR imaging under challenging environments.

Main Methods:

• A dual-modality system combining an SVE micro-attenuation camera and an event sensor in an asymmetric configuration.
• A two-stage cross-modal alignment framework using feature-guided homography estimation and multi-scale refinement.
• A cross-modal HDR reconstruction network employing convolutional fusion, mutual-information regularization, and a learnable fusion loss.

Main Results:

• The proposed system demonstrates consistent improvements in highlight recovery and edge fidelity compared to frame-only or event-only HDR methods.
• Experiments on synthetic and real data validate the system's robustness in highly dynamic and radiometrically challenging scenes.
• The co-designed system effectively handles non-coaxial geometry and heterogeneous optics through advanced alignment techniques.

Conclusions:

• Joint optimization of optical design, cross-modal alignment, and computational fusion is crucial for reliable HDR perception.
• The integrated SVE and event camera system offers a significant advancement for HDR imaging in extreme environments.
• This approach provides a robust foundation for future developments in challenging visual perception tasks.