Gradient methods for the optimization of dynamical systems containing neural networks

Area of Science:

• Artificial Intelligence
• Machine Learning
• Dynamical Systems Theory

Background:

• Backpropagation is a standard algorithm for training artificial neural networks.
• Optimizing weights in multilayer neural networks is crucial for their performance.
• Existing methods may face challenges with complex, interconnected systems.

Purpose of the Study:

• To introduce dynamic backpropagation, an extension of the backpropagation method.
• To provide a straightforward approach for optimizing neural network weights.
• To enable the gradient calculation for performance indices in nonlinear dynamical systems.

Main Methods:

• The study extends the backpropagation algorithm.
• It combines gradient methods from linear dynamical systems with neural network backpropagation.
• The method is applicable to systems composed of interconnected linear dynamical systems and multilayer neural networks.

Main Results:

• Dynamic backpropagation offers a method for optimizing weights in multilayer neural networks.
• It facilitates the gradient computation for performance indices in nonlinear dynamical systems.
• The approach is suitable for complex, interconnected systems.

Conclusions:

• Dynamic backpropagation is a viable extension of the backpropagation method.
• The technique simplifies the optimization of neural network parameters.
• Diagrammatic representations are emphasized for practical implementation.