Exploring and comparing the best "direct methods" for the efficient training of MLP-networks

Area of Science:

• Artificial Intelligence
• Machine Learning
• Numerical Methods

Background:

• Backpropagation algorithms face challenges with local minima and slow pattern adaptation during training.
• Existing methods often struggle with the computational demands of large-scale neural network training.

Purpose of the Study:

• To present a novel class of algorithms that address the limitations of traditional backpropagation.
• To evaluate the performance of specific direct numerical methods for weight matrix computation.

Main Methods:

• Investigated least-squares backpropagation (FBFBK-LSB) and iterative conjugate gradient singular-value decomposition (ICGSVD) algorithms.
• Employed direct numerical methods for efficient weight matrix computation.
• Conducted numerical experiments on several benchmark problems.

Main Results:

• The iterative conjugate gradient singular-value decomposition (ICGSVD) algorithm showed enhanced reliability and efficiency.
• FBFBK-LSB algorithm performance was also evaluated against ICGSVD.
• Numerical results confirmed the advantages of the proposed ICGSVD approach.

Conclusions:

• The developed ICGSVD algorithm offers a more reliable and efficient alternative to traditional backpropagation.
• Direct numerical methods provide a viable solution for overcoming common training difficulties in neural networks.
• The findings suggest potential for improved performance in complex machine learning tasks.