Automated learning for reducing the configuration of a feedforward neural network

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Cascade-correlation (CAS) algorithm incrementally adds hidden units, making final network size unpredictable.
• Restarting CAS training is necessary when the number of hidden units exceeds expectations.
• Efficiently determining the optimal number of hidden units in ANNs is a persistent challenge.

Purpose of the Study:

• To introduce two new learning mechanisms, Mixed-mode learning (MM) and Population-based learning for ANNs (PLAN), to improve ANN training.
• To reduce the number of hidden units required by the cascade-correlation (CAS) learning algorithm.
• To enhance classification performance and training efficiency in artificial neural networks.

Main Methods:

• Mixed-mode learning (MM) relaxes the learning objective by allowing many-to-many mappings, reducing epochs and hidden units.
• Population-based learning for ANNs (PLAN) dynamically selects promising network configurations at runtime based on error and time constraints.
• Both methods offer alternatives within the learning process, managed dynamically using runtime information.

Main Results:

• MM reduces learning epochs and the number of hidden units needed for convergence.
• PLAN avoids training unpromising networks by dynamically scheduling and selecting optimal configurations.
• Both mechanisms demonstrated effective performance on the two-spiral, two-region classification, and Pima Indian diabetes diagnosis problems.

Conclusions:

• MM and PLAN offer effective strategies for optimizing ANN training, particularly with the CAS algorithm.
• These mechanisms address the challenge of unpredictable hidden unit requirements in incremental learning.
• The proposed methods enhance efficiency and performance in solving complex binary classification problems.