E-3DGS: 3D Gaussian splatting with exposure and motion events

Area of Science:

• Computer Vision and Imaging

Background:

• Standard frame-based cameras struggle with 3D reconstruction in real-world conditions due to motion blur and poor illumination.
• Event cameras offer potential but require specialized methods for high-quality 3D reconstruction.

Purpose of the Study:

• To develop an enhanced 3D reconstruction framework using event cameras capable of capturing both motion and exposure events.
• To improve the quality and robustness of 3D reconstruction under challenging environmental conditions.

Main Methods:

• Incorporated a hardware-level transmittance adjustment device with event cameras.
• Utilized motion events for fast-motion scenarios and exposure events for high-quality grayscale reconstruction.
• Developed a framework supporting high-quality, fast, and hybrid 3D reconstruction modes, optimizing event-based 3D Gaussian splatting (3DGS).

Main Results:

• Exposure events significantly improved fine detail reconstruction compared to motion events on the EventNeRF dataset.
• The proposed method outperformed frame-based cameras in low illumination and overexposure scenarios.
• Introduced EME-3D, a new real-world dataset for event-based 3D reconstruction.

Conclusions:

• The E-3DGS framework achieves faster and higher-quality 3D reconstruction than existing event-based methods.
• The approach is more cost-effective than combined event and RGB data methods.
• This work establishes a new benchmark for robust and efficient event-based 3D reconstruction in challenging conditions.