Stability Analysis of the Modified Levenberg-Marquardt Algorithm for the Artificial Neural Network Training

Area of Science:

• Artificial Intelligence
• Machine Learning
• Neural Networks

Background:

• Levenberg-Marquardt and Newton algorithms utilize the Hessian matrix for artificial neural network (ANN) learning.
• Existing algorithms like Levenberg-Marquardt and Newton face challenges with singularity points in learning rates and employ multiple learning rates, potentially affecting error stability and weight boundedness.

Purpose of the Study:

• To propose a modified Levenberg-Marquardt algorithm for ANN learning that ensures error stability and weights boundedness.
• To address the limitations of singularity points in learning rates and the use of multiple learning rates found in conventional Levenberg-Marquardt and Newton algorithms.

Main Methods:

• Development of a modified Levenberg-Marquardt algorithm for ANN training and testing.
• The modified algorithm eliminates singularity points in learning rates and uses a single learning rate.
• Lyapunov techniques are employed to guarantee error stability and weights boundedness of the proposed algorithm.

Main Results:

• The modified Levenberg-Marquardt algorithm demonstrates improved error stability and weights boundedness compared to standard algorithms.
• Performance comparison of the modified Levenberg-Marquardt algorithm against Levenberg-Marquardt, Newton, and stable gradient algorithms using electric and brain signals datasets.

Conclusions:

• The modified Levenberg-Marquardt algorithm offers a robust and stable approach for artificial neural network learning.
• The proposed method effectively handles training and testing stages while maintaining desirable properties like error stability and bounded weights, outperforming traditional methods on real-world datasets.