HeteroFusion: Dense Scene Reconstruction Integrating Multi-Sensors

Area of Science:

• Robotics
• Computer Vision
• 3D Reconstruction

Background:

• Current 3D reconstruction methods often rely on single RGBD cameras, limiting accuracy due to tracking failures in areas with insufficient features.
• Sensor fusion offers a promising avenue to enhance self-localization robustness and accuracy by combining diverse sensor strengths.

Purpose of the Study:

• To develop a novel, multi-sensor data integration approach for real-time, dense 3D reconstruction of indoor scenes.
• To improve the robustness and accuracy of 3D reconstruction by overcoming limitations of single-sensor systems.

Main Methods:

• Integration of data from 2D range sensors, inertial measurement units (IMU), and wheel encoders for enhanced tracking.
• Development of a 2D truncated signed distance field (TSDF) volume representation for laser frame integration and ray-casting.
• Implementation of a unified cost function for pose estimation and a classifier for validating poses in loop-closure optimization.

Main Results:

• The proposed multi-sensor fusion method significantly improves tracking robustness and 3D reconstruction quality.
• Experimental evaluations on real-world and synthetic data demonstrate superior performance compared to state-of-the-art RGBD and LiDAR systems.
• The system achieves robust acquisition of dense 3D reconstructions in challenging indoor environments.

Conclusions:

• Multi-sensor fusion is a viable strategy for achieving robust and accurate real-time 3D reconstruction in indoor environments.
• The developed approach offers a significant advancement over existing single-sensor methods, particularly in scenarios with limited geometric features.
• This work paves the way for more reliable robotic perception and mapping systems in complex indoor settings.