Use of a quasi-Newton method in a feedforward neural network construction algorithm

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Feedforward neural networks are widely used in machine learning.
• Constructing efficient neural networks with optimal hidden layer sizes remains a challenge.
• Existing methods for neural network training can be computationally intensive.

Purpose of the Study:

• To present a novel algorithm for constructing single hidden layer feedforward neural networks.
• To incorporate the quasi-Newton method for efficient error function minimization during network growth.
• To evaluate the algorithm's efficiency, robustness, and performance on benchmark problems.

Main Methods:

• Development of a constructive algorithm for feedforward neural networks.
• Utilization of the quasi-Newton method to iteratively minimize error functions.
• Testing the algorithm on the n-bit parity problem and the breast cancer diagnosis dataset.

Main Results:

• The algorithm efficiently constructed neural networks with fewer hidden units than 'n' for the n-bit parity problem (n=4-7).
• For breast cancer diagnosis, the constructed neural networks achieved high accuracy on both training and testing datasets.
• The algorithm demonstrated efficiency and robustness in experimental evaluations.

Conclusions:

• The proposed algorithm offers an efficient and robust method for constructing single hidden layer feedforward neural networks.
• The approach effectively balances network complexity (hidden units) with high predictive accuracy.
• This constructive algorithm shows promise for applications in pattern recognition and diagnostic tasks.