Spatio-temporal transformer traffic prediction network based on multi-level causal attention.

Area of Science:

• Intelligent Transportation Systems
• Machine Learning
• Data Science

Background:

• Traffic prediction is crucial for intelligent transportation systems.
• Traffic flow data presents complex temporal and spatial dependencies, challenging accurate prediction.
• Existing models struggle with these complex spatiotemporal characteristics.

Purpose of the Study:

• To propose a novel spatiotemporal Transformer network for enhanced traffic prediction.
• To address the challenges posed by complex temporal and spatial dependencies in traffic flow data.
• To improve the accuracy and efficiency of traffic flow forecasting.

Main Methods:

• Developed a spatiotemporal Transformer network named MLCAFormer.
• Designed a multi-level temporal causal attention mechanism for hierarchical dependency capture.
• Introduced a node-identity-aware spatial attention mechanism for improved node distinction and spatial correlation learning.
• Integrated original traffic flow, cyclical patterns, and collaborative spatio-temporal embedding as input features.

Main Results:

• MLCAFormer demonstrated superior performance compared to benchmark models on four real-world traffic datasets (METR-LA, PEMS-BAY, PEMS04, PEMS08).
• The multi-level causal attention effectively captured long- and short-term temporal dependencies.
• The node-identity-aware spatial attention enhanced spatial correlation learning.

Conclusions:

• The proposed MLCAFormer network offers a significant advancement in traffic prediction accuracy.
• The integration of multi-level causal attention and node-identity-aware spatial attention is effective for handling complex spatiotemporal traffic data.
• MLCAFormer shows strong potential for real-world intelligent transportation applications.