Exploring constructive cascade networks

Area of Science:

• Machine Learning
• Artificial Intelligence
• Neural Networks

Background:

• Constructive algorithms are effective for training feedforward neural networks.
• Generalization is a critical property for these networks.

Purpose of the Study:

• To examine the effect of regularization on generalization in constructive cascade algorithms.
• To introduce and evaluate an adaptive method for setting regularization magnitude.

Main Methods:

• Empirical studies were conducted to analyze regularization techniques.
• Early stopping, cubic penalty terms, and an adaptive regularization method were investigated.
• The acasper algorithm was developed and compared to cascade correlation.

Main Results:

• Combining early stopping with regularization enhanced generalization compared to early stopping alone.
• A cubic penalty term benefited generalization in cascade networks.
• The adaptive regularization method achieved generalization similar to fixed settings and produced smaller networks for complex problems.

Conclusions:

• Regularization is crucial for improving generalization in constructive neural networks.
• The adaptive regularization method offers an efficient approach to network training.
• The acasper algorithm demonstrates strong generalization and network construction capabilities.