Topology-aware multi-information fusion for object recognition

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Multi-source information fusion is vital for object recognition.
• Occlusion and data inconsistencies in real-world scenarios challenge feature extraction reliability.

Purpose of the Study:

• To propose a Topology-Aware Multi-Information Fusion (TMF) model for improved robustness and generalizability in multi-sensor feature extraction.
• To enhance object recognition performance in challenging environments like autonomous driving and industrial inspection.

Main Methods:

• The TMF model integrates topological architectures into feature extraction and propagation.
• Key modules include the Enhancing Feature Module (EFM) for refining local geometric structures and the Attention Topology Module (ATM) for topology-aware attention during feature propagation.
• A fusion strategy concatenates 2D RGB features with 3D point-cloud features.

Main Results:

• The TMF model achieved significant improvements in mean Intersection over Union (mIoU) compared to PointNet, with a 15.3% increase on the S3DIS dataset and a 17.1% increase on the Semantic3D dataset.
• The model demonstrated effectiveness in integrating 3D point cloud and 2D image features, maximizing complementary advantages.
• Validation on self-collected real-world data confirmed the model's applicability across different data distributions.

Conclusions:

• The proposed TMF model offers a robust and generalizable solution for multi-modal object recognition.
• Integrating topological architectures enhances feature extraction and fusion, leading to superior performance in complex real-world applications.
• The TMF model shows strong potential for practical deployment in autonomous driving and industrial inspection.