Merging Back-propagation and Hebbian Learning Rules for Robust Classifications

Area of Science:

• Artificial Intelligence
• Machine Learning
• Neural Networks

Background:

• Multi-layer Perceptrons (MLPs) are widely used for classification tasks.
• Standard back-propagation algorithms can be sensitive to input variations, potentially limiting robustness and generalization.
• Improving the efficiency and robustness of neural network training is an ongoing research area.

Purpose of the Study:

• To develop a new learning algorithm for MLPs that enhances robustness and classification performance.
• To integrate Hebbian learning principles with error back-propagation.
• To reduce sensitivity to input variations and improve generalization capabilities.

Main Methods:

• A novel learning algorithm is proposed by imposing saturation requirements on hidden-layer neural activations.
• Derivatives of sigmoid functions at hidden layers are added to the standard output error with significance factors.
• The total error is minimized using the steepest-descent method, merging error back-propagation and Hebbian learning rules.

Main Results:

• The new algorithm demonstrates improved robustness and better generalization for classifier networks.
• Distributed representations at hidden layers are suppressed, leading to efficient hidden neuron utilization.
• Computer simulations show faster learning convergence and enhanced robustness for binary pattern classification and hetero-associations.

Conclusions:

• The proposed saturation-based learning algorithm offers a computationally inexpensive method to improve MLP robustness and generalization.
• This approach effectively combines error back-propagation with Hebbian learning principles.
• The method shows significant promise for enhancing neural network performance in classification and pattern association tasks.