PLM-Net: Perception Latency Mitigation Network for Vision-Based Lateral Control of Autonomous Vehicles

Area of Science:

• Robotics
• Computer Vision
• Machine Learning

Background:

• Perception latency, the delay in vision-based systems, significantly impacts lane-keeping performance and stability.
• Existing methods for latency compensation are limited in vision-based imitation learning.
• Deep learning approaches for autonomous driving need robust solutions for perception delays.

Purpose of the Study:

• Introduce the Perception Latency Mitigation Network (PLM-Net) to address perception latency in vision-based lane-keeping.
• Develop a modular framework that mitigates latency effects without altering the core control system.
• Evaluate the effectiveness of PLM-Net in reducing steering errors under various latency conditions.

Main Methods:

• PLM-Net employs a modular deep learning architecture with a frozen Base Model (BM) and a Timed Action Prediction Model (TAPM).
• TAPM predicts future steering actions based on discrete latency conditions.
• Real-time mitigation is achieved by interpolating model outputs according to measured latency, adapting to constant and time-varying delays.

Main Results:

• PLM-Net demonstrated significant reductions in steering error in simulation.
• Achieved up to 62% reduction in Mean Absolute Error (MAE) for constant latency.
• Achieved up to 78% reduction in MAE for time-varying latency.

Conclusions:

• PLM-Net offers an effective modular solution for mitigating perception latency in vision-based lane-keeping systems.
• The framework shows architectural feasibility for improving lateral control under controlled simulation conditions.
• Public release of code, data, and demonstrations facilitates further research and development.