Real-Time Hybrid Multi-Sensor Fusion Framework for Perception in Autonomous Vehicles

Area of Science:

• Robotics and Artificial Intelligence
• Computer Vision
• Sensor Fusion

Background:

• Existing sensor fusion frameworks often prioritize accuracy over implementation feasibility in real-world autonomous vehicles.
• High computational demands of some fusion architectures limit their deployment on embedded edge computers.

Purpose of the Study:

• To propose a cost-effective, lightweight, modular, and robust hybrid multi-sensor fusion pipeline for autonomous vehicle environment perception.
• To improve the implementation feasibility of sensor fusion in resource-constrained embedded systems.

Main Methods:

• Developed a hybrid fusion framework integrating an encoder-decoder based Fully Convolutional Neural Network (FCNx) with an Extended Kalman Filter (EKF).
• Utilized a sensor configuration of camera, LiDAR, and radar, optimized for each fusion method.
• Employed FCNx for improved road detection and EKF for nonlinear state estimation.

Main Results:

• The FCNx algorithm demonstrated improved road detection accuracy compared to benchmark models while maintaining real-time efficiency.
• The hybrid framework achieved better performance across various environmental scenarios compared to baseline networks in tests on over 3,000 road scenes.
• Real-time environment perception was successfully demonstrated through in-vehicle implementation and testing with actual sensor data.

Conclusions:

• The proposed hybrid sensor fusion system offers a practical and efficient solution for autonomous vehicle environment perception.
• The framework's modularity and robustness (in case of sensor failure) make it suitable for real-world autonomous driving applications.
• This approach balances high performance with the computational constraints of embedded automotive systems.