Convergence analysis of sliding mode trajectories in multi-objective neural networks learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Control Theory

Background:

• Neural network supervised learning involves a trade-off between data-set error and network complexity.
• Existing methods often focus on selecting a final optimal state, neglecting the learning process itself.

Purpose of the Study:

• To introduce Pareto-optimality for representing trade-offs in neural network learning.
• To present neural network learning as a dynamic system control problem.
• To enable control over learning trajectories for enhanced optimization.

Main Methods:

• Utilized the Pareto-optimality concept to define a set of trade-off solutions.
• Modeled neural networks as dynamic systems with error and complexity as state variables.
• Applied sliding mode dynamics to control learning trajectories in the state space.

Main Results:

• Demonstrated that arbitrary learning trajectories can be achieved by managing sliding mode gains.
• Provided formal proofs for convergence conditions of the proposed learning method.
• Showcased that learning trajectories can be individually assessed against additional objective functions.

Conclusions:

• The trajectory learning concept offers a more nuanced approach than simply selecting a final Pareto-optimal state.
• This framework allows for diverse learning pathways and individual state evaluation.
• The method provides formal guarantees for convergence in neural network training.