A Temporal Window Attention-Based Window-Dependent Long Short-Term Memory Network for Multivariate Time Series...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Science

Background:

• Multivariate time series prediction models struggle with complex, nonlinear interdependencies within temporal windows.
• Existing attention mechanisms often fail to capture global spatio-temporal patterns by not directly extracting time-step relevant features.

Purpose of the Study:

• To propose a novel Temporal Window Attention-based Window-Dependent Long Short-Term Memory network (TWA-WDLSTM) for enhanced multivariate time series prediction.
• To improve the capture of temporal dependencies and spatio-temporal patterns from a global perspective.

Main Methods:

• An encoder-decoder framework is utilized, incorporating a temporal window attention mechanism in the encoder to select relevant exogenous series.
• A window-dependent long short-term memory network (WDLSTM) is introduced for encoding input sequences and capturing long-term dependencies.
• WDLSTM is also employed in the decoder for generating prediction values.

Main Results:

• The proposed TWA-WDLSTM model demonstrated superior performance compared to various state-of-the-art models across four real-world datasets.
• The temporal window attention mechanism exhibited good interpretability, allowing for the identification of influential variables for future predictions.

Conclusions:

• TWA-WDLSTM effectively enhances temporal dependencies and outperforms existing models in multivariate time series prediction.
• The model's attention mechanism provides valuable insights into variable contributions, aiding in understanding prediction drivers.