Enhanced training algorithms, and integrated training/architecture selection for multilayer perceptron networks

Area of Science:

• Artificial Intelligence
• Machine Learning
• Neural Networks

Background:

• Standard backpropagation algorithm for multilayer perceptron (MLP) training exhibits slow asymptotic convergence.
• Need for improved training algorithms to accelerate MLP convergence and enhance performance.

Purpose of the Study:

• To develop and evaluate enhanced MLP training algorithms using nonlinear least-squares and quasi-Newton optimization techniques.
• To compare the effectiveness of these enhanced algorithms against the standard backpropagation algorithm.
• To present an integrated approach for simultaneous training and architecture selection.

Main Methods:

• Implementation of enhanced MLP training algorithms based on sophisticated nonlinear least-squares and quasi-Newton optimization.
• Comparative analysis of enhanced algorithms versus backpropagation on various benchmark problems.
• Development and testing of an integrated training and architecture selection strategy.

Main Results:

• Enhanced algorithms demonstrate superior convergence rates compared to standard backpropagation.
• The integrated approach effectively optimizes both MLP training and architecture selection.
• Successful application to synthetic and real-world pattern recognition tasks.

Conclusions:

• Nonlinear least-squares and quasi-Newton methods offer significant improvements in MLP training speed and efficiency.
• The integrated training and architecture selection approach enhances overall pattern recognition performance.
• These advanced techniques provide a more effective solution for complex machine learning problems.