Marginally Stable Triangular Recurrent Neural Network Architecture for Time Series Prediction

Area of Science:

• Artificial Intelligence
• Machine Learning
• Dynamical Systems

Background:

• Recurrent neural networks (RNNs) are powerful tools for sequence modeling.
• Ensuring training stability and preventing overfitting in RNNs remains a challenge.
• Existing architectures often require complex training procedures.

Purpose of the Study:

• Introduce a novel discrete-time recurrent neural network architecture.
• Simplify the process of ensuring network and training stability.
• Enhance the learning ability for time-series prediction tasks.

Main Methods:

• Utilized triangular feedback weight matrices with block-diagonal elements.
• Ensured eigenvalues lie on the unit circle by updating weights based on angular error.
• Implemented a dual triangular architecture with constrained block-diagonal weights to prevent overfitting.
• Employed input parsing and selective application to subnetworks.

Main Results:

• Demonstrated high effectiveness in time-series prediction for nonlinear and chaotic systems.
• Achieved simplified training processes while maintaining learning stability.
• The modular architecture mimics fully recurrent networks.
• Enhanced learning performance through selective input application.

Conclusions:

• The proposed discrete-time recurrent neural network architecture offers a stable and simplified approach to training.
• It shows significant promise for time-series prediction in complex dynamic systems.
• The modular and dual triangular design contributes to enhanced learning and reduced overfitting.