Event fusion photometric stereo network.

Area of Science:

• Computer Vision
• Computational Imaging
• Robotics

Background:

• Photometric stereo traditionally uses RGB cameras in controlled, dark environments.
• Ambient illumination challenges traditional methods due to limited RGB camera dynamic range.
• Accurate surface normal estimation is crucial for 3D reconstruction and analysis.

Purpose of the Study:

• To develop a novel method for estimating surface normals under ambient illumination using RGB and event cameras.
• To leverage the high dynamic range of event cameras to overcome limitations of RGB cameras in photometric stereo.
• To improve the accuracy and robustness of photometric stereo in unconstrained lighting conditions.

Main Methods:

• Proposed the Event Fusion Photometric Stereo Network (EFPS-Net) integrating RGB and event camera data.
• Developed an event interpolation technique to enhance light information capture.
• Created fused RGB-event observation maps for surface normal estimation.

Main Results:

• EFPS-Net significantly outperforms state-of-the-art methods relying solely on RGB frames.
• Achieved a 7.94% reduction in mean average error compared to existing methods.
• Demonstrated effective surface normal estimation in diverse ambient light environments.

Conclusions:

• EFPS-Net offers a robust solution for photometric stereo under challenging ambient lighting.
• The fusion of RGB and event camera data enhances surface normal estimation accuracy.
• A new dataset for ambient light photometric stereo was curated, facilitating future research.