MixedFusion: Real-Time Reconstruction of an Indoor Scene with Dynamic Objects

Area of Science:

• Computer Vision
• Robotics
• 3D Reconstruction

Background:

• Current indoor scene reconstruction methods primarily address static environments.
• Real-world indoor scenes frequently contain dynamic elements like people and moving objects.
• Accurate reconstruction of dynamic scenes remains a significant challenge in real-time applications.

Purpose of the Study:

• To develop an end-to-end system for real-time 3D indoor scene reconstruction that incorporates dynamic objects.
• To address the limitations of existing methods by enabling the fusion of both static and dynamic scene components.
• To improve the applicability of 3D reconstruction techniques in realistic, dynamic indoor environments.

Main Methods:

• An end-to-end system utilizing a depth sensor for on-the-fly scene scanning.
• A novel Sigmoid-based Iterative Closest Point (S-ICP) method to decouple camera and scene motion.
• Segmentation of scenes into static and dynamic parts for differential motion estimation.
• Graph node-based motion representation and model-to-depth fitting for dynamic object motion reconstruction.
• A mixed voxel allocation scheme for efficient fusion of static and dynamic scene elements.

Main Results:

• Successful real-time reconstruction and fusion of 3D geometry for both static and dynamic objects within indoor scenes.
• Effective decoupling of camera ego-motion from object motion in dynamic environments.
• Demonstrated ability to handle common dynamic elements such as moving people and floating curtains.
• Validation through experimental results showing the system's robustness and accuracy.

Conclusions:

• The proposed system significantly advances real-time indoor scene reconstruction by effectively integrating dynamic object handling.
• This technique extends the capabilities of current 3D reconstruction methods, making them suitable for more complex and realistic scenarios.
• The developed approach provides a robust solution for fusing static and dynamic scene geometry, paving the way for broader applications in robotics and augmented reality.