Approximate Bayesian MLP regularization for regression in the presence of noise

Area of Science:

• Machine Learning
• Artificial Intelligence
• Statistical Modeling

Background:

• Multilayer perceptrons (MLPs) are widely used for regression tasks.
• Existing regularization methods often assume smooth regression functions, limiting their applicability.
• Handling noisy and non-smooth data remains a challenge in neural network training.

Purpose of the Study:

• To develop a novel regularization method for MLPs capable of learning regression functions with noise and discontinuities.
• To address the limitations of existing methods that assume function smoothness.
• To improve the accuracy of MLPs in modeling complex, non-smooth relationships.

Main Methods:

• A Bayesian approach is employed to identify non-smooth data within the training set.
• A probability distribution is utilized to detect discontinuities in the regression function.
• A proposed objective function incorporates identified non-smooth data to guide MLP fitting.

Main Results:

• The proposed regularization method enables MLPs to accurately fit non-smooth regression functions.
• Experimental simulations demonstrate superior performance compared to existing MLP training methods.
• The method shows effectiveness even in the presence of significant noise.

Conclusions:

• The novel regularization technique enhances MLP performance on non-smooth regression tasks.
• This approach offers a robust solution for modeling complex data without prior smoothness assumptions.
• The methodology is compatible with deep learning models, broadening its potential applications.