Learning Temporal Features With Alternated Similarity and Proximity Attention for Time-Series Prediction.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Science

Background:

• Time-series prediction is crucial across many scientific fields.
• Attention-based models are promising for long-term forecasting.
• Vanilla attention has limitations in capturing complex dependencies.

Purpose of the Study:

• To propose a novel attention mechanism for enhanced time-series forecasting.
• To address the limitations of vanilla attention in capturing high-order dependencies.
• To improve the accuracy of long-term time-series predictions.

Main Methods:

• Introduced Alternated Similarity And Proximity Attention (ASAP-attention).
• ASAP-attention uses random walks on two graphs: one for feature similarity and one for temporal proximity.
• The model alternately visits these graphs, leveraging previous states for coherent predictions.
• Integrated ASAP-attention with an encoder-only Transformer architecture.

Main Results:

• ASAP-attention demonstrated superior performance against state-of-the-art methods.
• Achieved highly promising results on diverse benchmark datasets for long time-series forecasting.
• The method effectively captures implicit and heterogeneous data dependencies without positional encoding.

Conclusions:

• ASAP-attention offers a powerful new approach for long-term time-series forecasting.
• The proposed method enhances the ability to model complex temporal and feature interactions.
• ASAP-attention shows significant potential for applications in weather, finance, and healthcare forecasting.