Incorporating multi-path risk assessment in transformer-based pedestrian crossing action prediction.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Transportation Safety

Background:

• Predicting pedestrian crossing actions is crucial for autonomous driving and traffic safety.
• Existing methods often struggle with complex pedestrian-vehicle interactions (PVIs) and diverse scenarios.
• Accurate short-term prediction of pedestrian behavior is essential for preventing collisions.

Purpose of the Study:

• To develop a novel Transformer-based framework for predicting pedestrian crossing actions.
• To visualize and encode multi-path pedestrian-vehicle collision risks into informative RGB images.
• To improve the accuracy and stability of pedestrian action prediction in dynamic traffic environments.

Main Methods:

• A two-step framework involving risk image generation and multi-modal fusion using a Cross-Attention Transformer.
• Encoding spatio-temporal PVI risks into high-density RGB images.
• Integrating risk images with historical sequential data (pedestrian action, vehicle velocity) for prediction.

Main Results:

• Risk images effectively capture significant spatiotemporal features and transition dynamics in PVIs.
• The Transformer architecture demonstrates efficacy in processing multi-path risk images.
• Achieved 87.34% prediction accuracy for 0.5s ahead, maintaining stability up to 2s, and improving detection of abrupt action changes.

Conclusions:

• The proposed Transformer framework offers a robust method for predicting pedestrian crossing actions based on visualized collision risks.
• The risk image representation provides valuable safety information for PVI analysis.
• This approach enhances the reliability of autonomous systems in complex road scenarios.