Attention-Linear Trajectory Prediction

Recently, a large number of Transformer-based solutions have emerged for the trajectory prediction task, but there are shortcomings in the effectiveness of Transformers in trajectory prediction. Specifically, while position encoding preserves some of the ordering information, the self-attention mechanism at the core of the Transformer has its alignment invariance that leads to the loss of temporal information, which is crucial for trajectory prediction. For this reason, we design a simple and efficient strategy for temporal information extraction and prediction of trajectory sequences using the self-attention mechanism and linear layers. The experimental results show that the strategy can improve the average accuracy by 15.31%, effectively combining the advantages of the linear layer and the self-attention mechanism, while compensating for the shortcomings of the Transformer. Additionally, we conducted an empirical study to explore the effectiveness of the linear layer and sparse self-attention mechanisms in trajectory prediction.