A Comparative Analysis between Efficient Attention Mechanisms for Traffic Forecasting without Structural Priors.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Dot-product attention is crucial for contextual information in AI models.
• Quadratic complexity of attention maps limits scalability.
• Efficient attention alternatives are emerging.

Purpose of the Study:

• Compare efficient attention mechanisms against baseline dot-product attention.
• Evaluate performance in a spatio-temporal traffic prediction model.
• Quantify improvements in training and inference times.

Main Methods:

• Implemented a purely attention-based spatio-temporal forecasting model.
• Integrated and analyzed several efficient attention mechanisms.
• Conducted comparative experiments on traffic prediction datasets.

Main Results:

• Efficient attention reduced training times by up to 28%.
• Inference times were decreased by up to 31%.
• Model performance remained comparable to the baseline.

Conclusions:

• Efficient attention mechanisms offer significant speedups for traffic prediction.
• These methods provide a viable alternative to dot-product attention without sacrificing accuracy.
• The findings are applicable to other spatio-temporal forecasting tasks.