A new adaptive backpropagation algorithm based on Lyapunov stability theory for neural networks

Area of Science:

• Neural Networks
• Control Theory
• Adaptive Systems

Background:

• Traditional backpropagation (BP) algorithms rely on gradient descent, which can be slow and may get stuck in local minima.
• Ensuring stability and convergence in neural networks is crucial for reliable performance, especially in dynamic systems.
• Input disturbances can significantly impact the accuracy and stability of neural network outputs.

Purpose of the Study:

• To develop a new adaptive backpropagation (BP) algorithm grounded in Lyapunov stability theory.
• To demonstrate the algorithm's ability to achieve asymptotic convergence of tracking errors in neural networks.
• To enhance the robustness of neural networks against bounded input disturbances.

Main Methods:

• A Lyapunov function candidate V(k) is selected to represent the tracking error.
• Neural network weights are adaptively updated from the output to the input layer to ensure deltaV(k) < 0.
• The algorithm constructs an energy surface with a single global minimum for weight adjustment.

Main Results:

• The proposed Lyapunov adaptive BP algorithm guarantees asymptotic convergence of the output tracking error to zero.
• The algorithm effectively eliminates the effects of bounded input disturbances.
• Simulation results for an adaptive filter show fast error convergence and strong robustness.

Conclusions:

• The Lyapunov adaptive BP algorithm provides a stable and robust method for neural network training.
• This approach offers an alternative to gradient-based methods, focusing on energy surface construction for global convergence.
• The algorithm demonstrates significant potential for applications requiring high accuracy and resilience, such as adaptive filtering.