A panoramic driving perception fusion algorithm based on multi-task learning.

Area of Science:

• * Intelligent Transportation Systems
• * Computer Vision
• * Sensor Fusion

Background:

• * Intelligent connected vehicles require advanced driver assistance systems (ADAS) with robust perception capabilities.
• * Current ADAS often face hardware limitations, processing single-sensor and single-task data, hindering complex panoramic perception.
• * Existing algorithms like YOLOP show promise in multi-task learning but struggle with feature map pooling and detail loss.

Purpose of the Study:

• * To develop a novel panoramic driving perception fusion algorithm leveraging multi-task learning.
• * To enhance the processing of multi-sensor (lidar and vision) and multi-task data for improved driving perception.
• * To overcome the limitations of existing methods regarding feature map adaptability and detail preservation.

Main Methods:

• * Proposed a multi-task learning fusion algorithm incorporating diverse loss functions.
• * Implemented specialized processing steps for lidar point cloud data.
• * Fused perception information from lidar and vision sensors for synchronized multi-task, multi-sensor data processing.

Main Results:

• * The proposed algorithm demonstrated superior performance in lane detection, drivable area detection, and vehicle detection compared to the YOLOP model on the BDD100K dataset.
• * Achieved an 11.6% improvement in lane detection accuracy.
• * Increased mean Intersection over Union (mIoU) for drivable area detection by 2.1% and mean Average Precision at 50% IoU (mAP50) for vehicle detection by 3.7%.

Conclusions:

• * The developed multi-task learning fusion algorithm effectively addresses the challenges of complex panoramic driving perception.
• * Synchronized processing of multi-sensor and multi-task data significantly enhances system performance and reliability.
• * The approach offers a viable solution for improving driver assistance systems in intelligent connected vehicles.