Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Significant advances in object detection, localization, and tracking exist for general computer vision tasks.
• Existing methods lack real-time capabilities for detecting, localizing, and tracking objects specifically in road environments.

Purpose of the Study:

• To develop a deep learning-based multi-object detection and tracking technique for road smart mobility.
• To address the real-time constraints inherent in autonomous driving and intelligent transportation systems.

Main Methods:

• An adapted YOLOv3 detector for object detection in the road context.
• An adaptive method for extracting 3D depth information using Monodepth2 (monocular) and MADNEt (stereoscopic) approaches.
• An improved SORT (Simple Online and Realtime Tracking) approach for 3D object tracking, incorporating an extended Kalman filter for enhanced position estimation.

Main Results:

• Comparative evaluation of monocular and stereoscopic depth estimation methods to identify the best real-time solution.
• Demonstration of a novel 3D object tracking approach that utilizes detection and distance estimation for initialization, improving upon traditional methods.
• Extensive experiments on the KITTI dataset showing superior performance compared to state-of-the-art approaches.

Conclusions:

• The proposed deep learning technique effectively addresses real-time multi-object detection, localization, and tracking in road environments.
• The integration of advanced depth estimation and improved 3D tracking enhances the safety and efficiency of smart mobility systems.
• The method shows significant potential for real-world applications in autonomous driving and intelligent transportation.