MLPNN Training via a Multiobjective Optimization of Training Error and Stochastic Sensitivity.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Neural Networks

Background:

• Multilayer perceptron neural network (MLPNN) training involves minimizing errors and penalty terms for generalization.
• Existing smoothness control methods like weight norm or Vapnik-Chervonenkis dimension have limitations in distinguishing MLPs.

Purpose of the Study:

• To propose a new penalty term for MLPNN training using a stochastic sensitivity measure (ST-SM).
• To enhance the generalization capabilities of MLPs by directly measuring output fluctuations.

Main Methods:

• Introduced the stochastic sensitivity measure (ST-SM) as a penalty term for MLPNN training.
• Developed a two-phase Pareto-based multiobjective training algorithm to minimize training error and ST-SM.
• Evaluated the method on 20 UCI datasets.

Main Results:

• MLPs trained with the proposed ST-SM penalty term and algorithm demonstrated improved accuracy on testing data.
• The ST-SM provides a direct measurement of MLP output fluctuations, indicating network smoothness.
• The proposed method outperformed several classical and recent MLPNN training techniques.

Conclusions:

• The stochastic sensitivity measure (ST-SM) is an effective penalty term for improving MLPNN generalization.
• The Pareto-based multiobjective training algorithm successfully optimizes both training error and network smoothness.
• This approach offers a more direct and effective way to enhance MLP performance on unseen data.