BuildingFusion: Semantic-Aware Structural Building-Scale 3D Reconstruction.

Area of Science:

• Computer Vision
• Robotics
• 3D Reconstruction

Background:

• Scalable geometry reconstruction and understanding remain significant challenges.
• Existing methods struggle with false loop closures in scenes with similar rooms and lack real-time semantic understanding.
• Online scene understanding and robust loop closure detection are crucial for large-scale reconstruction.

Purpose of the Study:

• To develop a semantic-aware structural building-scale reconstruction system called BuildingFusion.
• To enable collaborative, dense reconstruction of building-scale environments with on-the-fly semantic and structural information.
• To enhance robustness against false loop closures in visually similar environments.

Main Methods:

• A novel semantic-aware room-level loop closure detection (LCD) method leveraging instance-level embeddings from a 3D convolutional network.
• A graph matching module for geometry verification following semantic similarity measurement.
• A centralized architecture for collaborative scanning, with server-side room-level LCD for merging partial reconstructions.

Main Results:

• The semantic-aware room-level LCD significantly outperforms traditional image-based LCD in robustness.
• BuildingFusion demonstrates superior performance in extensive comparisons against existing methods.
• Live demonstrations confirm the feasibility of real-time, collaborative, and robust building-scale reconstruction.

Conclusions:

• BuildingFusion provides a robust and scalable solution for building-scale 3D reconstruction.
• Semantic information is a powerful feature for accurate place recognition and loop closure detection.
• The proposed system enables collaborative and real-time scene understanding for large environments.