The annealing robust backpropagation (ARBP) learning algorithm

Area of Science:

• Artificial Intelligence
• Machine Learning
• Neural Networks

Background:

• Multilayer feedforward neural networks are universal approximators but struggle with noisy training data.
• Traditional backpropagation is sensitive to outliers, and existing robust methods face initialization challenges.
• M-estimators in robust learning use loss functions to mitigate outlier effects, but discrimination can be imperfect.

Purpose of the Study:

• To propose a novel annealing robust backpropagation (ARBP) learning algorithm.
• To address the limitations of existing robust learning algorithms in handling data with outliers.
• To enhance the performance of neural networks trained on noisy datasets.

Main Methods:

• Introducing the annealing concept into robust backpropagation learning.
• Developing the annealing robust backpropagation (ARBP) algorithm.
• Experimentally determining the optimal annealing schedule (k/t) for best performance.

Main Results:

• The proposed ARBP algorithm demonstrates superior performance compared to other robust learning algorithms.
• ARBP effectively models data even in the presence of significant outliers.
• The annealing schedule k/t was identified as the most effective for ARBP.

Conclusions:

• The ARBP algorithm offers a robust solution for training neural networks with outlier-corrupted data.
• The integration of annealing significantly improves the discrimination against outliers.
• ARBP provides a more reliable and effective approach to neural network learning under noisy conditions.