Attention-Linear Trajectory Prediction.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Transformer models are increasingly used for trajectory prediction.
• Existing Transformer approaches suffer from temporal information loss due to self-attention's alignment invariance.
• Position encoding partially addresses ordering but not the core issue.

Purpose of the Study:

• To develop an effective strategy for temporal information extraction in trajectory prediction.
• To enhance the performance of Transformer-based trajectory prediction models.
• To investigate the combined effectiveness of linear layers and self-attention.

Main Methods:

• Designed a novel strategy combining self-attention mechanisms and linear layers.
• Focused on extracting and preserving temporal information crucial for trajectory prediction.
• Conducted empirical studies on linear layers and sparse self-attention.

Main Results:

• Achieved an average accuracy improvement of 15.31% in trajectory prediction.
• Demonstrated effective synergy between linear layers and self-attention mechanisms.
• Validated the strategy's ability to compensate for Transformer shortcomings.

Conclusions:

• The proposed strategy significantly enhances trajectory prediction accuracy.
• Combining linear layers and self-attention is a viable approach to overcome Transformer limitations.
• Further exploration of sparse self-attention is beneficial for this task.