Training feed-forward neural networks with gain constraints

Area of Science:

• Machine Learning
• Neural Networks
• Control Theory

Background:

• Inaccurate input-output gains in neural networks arise from correlated inputs or imperfect data, hindering optimization and control.
• Accurate gains are critical for the reliable performance of neural network models in various applications.

Purpose of the Study:

• To develop and evaluate a method for training feedforward neural networks with constrained input-output gains.
• To enhance the accuracy, robustness, and generalization capabilities of neural network models.

Main Methods:

• Implementing gain constraints as penalty terms within the objective function.
• Utilizing gradient descent for network training with adaptive and robust procedures for term balancing.
• Employing extrapolation training to extend the model's domain of validity.

Main Results:

• Demonstrated successful application on both artificial and real-world datasets.
• Achieved significant improvements in model generalization and accuracy through constrained gain training.
• The method proved effective in balancing objective function terms, even with inconsistent data.

Conclusions:

• The developed method effectively trains neural networks with accurate and constrained input-output gains.
• Constrained training and extrapolation significantly improve model generalization and reliability.
• The approach is robust, adaptable, and has practical applications in commercial models.