Robot Object Detection and Tracking Based on Image-Point Cloud Instance Matching.

Area of Science:

• Robotics and Autonomous Systems
• Computer Vision
• Sensor Fusion

Background:

• Mobile robot environmental perception relies on fusing diverse sensor data.
• Integrating semantic image information with precise LiDAR geometric data presents a significant challenge.
• Existing methods often struggle with efficient alignment and unified modeling of heterogeneous sensory inputs.

Purpose of the Study:

• To propose a highly extensible instance-aware fusion framework for mobile robot perception.
• To enable efficient alignment and unified modeling of RGB images and LiDAR point clouds.
• To improve multi-object tracking accuracy and robustness in complex environments.

Main Methods:

• Utilized an instance segmentation network for semantic mask extraction from RGB images.
• Employed a projection mechanism for spatial correspondence between image pixels and LiDAR points.
• Implemented 3D bounding box reconstruction via point cloud clustering and geometric fitting, with reprojection-based validation.
• Integrated a data association module and Kalman filter for closed-loop multi-object tracking.

Main Results:

• Achieved strong 2D and 3D detection performance on the KITTI dataset.
• Attained a Multi-Object Tracking Accuracy (MOTA) of 47.8 and an IDF1 score of 71.93.
• Demonstrated an average end-to-end latency of 173.9 ms in real-world experiments.
• Ablation studies confirmed the effectiveness of individual system components.

Conclusions:

• The proposed framework offers robust geometric reconstruction accuracy and tracking stability.
• Its lightweight design and low latency meet practical robotic deployment requirements.
• The system effectively fuses heterogeneous sensory data for advanced mobile robot perception and tracking.