Related Experiment Video
Updated: Jan 13, 2026
12:49
A Method for 3D Reconstruction and Virtual Reality Analysis of Glial and Neuronal Cells
Published on: September 28, 2019
13.3K
E-Sem3DGS: Monocular Human and Scene Reconstruction via Event-Aided Semantic 3DGS
Xiaoting Yin1, Hao Shi1,2, Kailun Yang3
1State Key Laboratory of Extreme Photonics and Instrumentation, National Engineering Research Center of Optical Instrumentation, Zhejiang University, Hangzhou 310027, China.
Sensors (Basel, Switzerland)
|January 10, 2026
Summary
This study introduces E-Sem3DGS, a novel framework using event and intensity cameras to reconstruct 3D humans and scenes from motion-blurred videos. It significantly enhances reconstruction quality and detail, overcoming limitations of prior methods.
Area of Science:
- Computer Vision
- Neural Rendering
- 3D Reconstruction
Background:
- Reconstructing animatable humans and scenes from monocular, motion-blurred videos is a significant challenge for current neural rendering techniques.
- Existing methods often fail under motion blur and provide incomplete scene modeling, focusing primarily on foreground human reconstruction.
- Event cameras offer high temporal resolution and motion blur robustness, complementing standard video sensors.
Purpose of the Study:
- To develop a novel framework, E-Sem3DGS, for joint 3D reconstruction of human avatars and static scenes from hybrid event-intensity streams.
- To address the limitations of existing methods in handling motion blur and achieving complete scene modeling.
- To leverage semantic attributes within 3D Gaussians for effective separation of dynamic and static content.
Main Methods:
- E-Sem3DGS employs a semantically augmented 3D Gaussian Splatting framework utilizing hybrid event and intensity data.
- It initializes human and scene Gaussians using Skinned Multi-Person Linear (SMPL) model priors and scene sampling, respectively, optimizing geometry, appearance, and semantics.
- Optical flow derived from events is used to supervise image-based optical flow, enforcing temporal coherence and mitigating motion blur.
Main Results:
- E-Sem3DGS significantly improves reconstruction quality on motion-blurred datasets, achieving a +49.7% PSNR increase on ZJU-MoCap-Blur (21.75 to 32.56).
- The method enhances both human avatar and background scene reconstruction accuracy and detail.
- Performance improvements were also demonstrated on the MMHPSD-Blur dataset, with a +13.48% PSNR increase (25.23 to 28.63).
Conclusions:
- E-Sem3DGS effectively reconstructs animatable humans and static scenes from motion-blurred videos using hybrid event-intensity data.
- The semantic augmentation of 3D Gaussians and event-derived optical flow are key to overcoming motion blur challenges.
- This framework represents a significant advancement in 3D reconstruction from challenging real-world video data.
