An equalized error backpropagation algorithm for the on-line training of multilayer perceptrons.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Neuroscience

Background:

• Training multilayer perceptrons (MLPs) with error backpropagation (EBP) can be time-consuming, requiring numerous epochs.
• On-line training paradigms reduce EBP time but can remain excessive for large networks and datasets.

Purpose of the Study:

• Introduce a novel on-line training algorithm, equalized error backpropagation (EEBP).
• Improve the speed, accuracy, and robustness of MLP training.
• Address challenges associated with training large neural networks on extensive data.

Main Methods:

• Developed the equalized error backpropagation (EEBP) algorithm.
• Incorporated weight-specific learning rates.
• Derived learning rate magnitudes from computable network and data properties.

Main Results:

• EEBP demonstrates improved training accuracy compared to standard EBP.
• The algorithm significantly reduces the number of training epochs required.
• EEBP exhibits enhanced robustness, particularly with badly scaled input data.

Conclusions:

• EEBP offers a more efficient and effective method for training MLPs.
• The use of adaptive, weight-specific learning rates is key to EEBP's performance.
• EEBP is a promising advancement for large-scale neural network training.