Deep context-attentive transformer transfer learning for financial forecasting | JoVE Visualize

Area of Science:

Computational finance
Machine learning for time series
Deep learning for financial markets

Background:

Financial time-series forecasting is crucial for investment decisions.
Existing deep learning models face challenges in capturing complex temporal dynamics and generalizing across markets.
The need for robust models that can adapt to varying market conditions is paramount.

Purpose of the Study:

To develop and evaluate 2CAT (CNN-Correlation-based Attention Transformer), a novel deep learning model for financial time-series forecasting.
To assess the effectiveness of 2CAT's integrated architecture, including signal decomposition, convolutional layers, and correlation-based attention.
To investigate the benefits of a transfer learning framework for enhancing cross-market generalization.

Main Methods:

Developed 2CAT, integrating CNNs, correlation-based attention, and transformers.
Employed a transfer learning framework involving pretraining, encoder freezing, and fine-tuning.
Evaluated the model on six major stock indices: DJIA, N225, HSI, SSE, BSE, and SET.
Conducted ablation studies and prediction horizon analyses.

Main Results:

2CAT demonstrated superior predictive accuracy on the Dow Jones Industrial Average (DJIA) compared to Deep-Transformer (e.g., R² of 0.9169 vs. 0.8274).
Achieved significant improvements on the Stock Exchange of Thailand (SET) index (R² of 0.9094), a market previously challenging for forecasting models.
Wilcoxon signed-rank tests confirmed statistically significant performance gains in both non-transfer and transfer learning scenarios (p < 0.05).
Transfer learning experiments validated the feasibility and effectiveness of cross-market knowledge transfer.

Conclusions:

2CAT is a robust and adaptable deep learning framework for financial time-series forecasting.
The model's architecture effectively captures temporal patterns and generalizes well across different financial markets.
Transfer learning significantly enhances forecasting performance and adaptability to diverse market conditions.