Optimal Control Theoretic Neural Optimizer: From Backpropagation to Dynamic Programming

Area of Science:

• Artificial Intelligence
• Machine Learning
• Control Theory

Background:

• Deep neural networks (DNNs) are crucial for AI advancements, but their optimization is complex.
• DNNs can be viewed as dynamical systems, enabling analysis via optimal control.
• Existing optimization methods like Backpropagation have algorithmic similarities to dynamic programming.

Purpose of the Study:

• To explore the algorithmic connection between Backpropagation and dynamic programming.
• To develop a new class of DNN optimization methods based on higher-order Bellman equation expansions.
• To introduce the Optimal Control Theoretic Neural Optimizer (OCNOpt).

Main Methods:

• Interpreting DNNs as dynamical systems within Optimal Control Programming.
• Leveraging the variational structure of Backpropagation.
• Applying higher-order expansions of the Bellman equation for optimization.
• Developing the OCNOpt algorithm.

Main Results:

• OCNOpt demonstrates improved robustness and efficiency compared to existing methods.
• The optimizer maintains manageable computational complexity.
• OCNOpt enables novel applications like layer-wise feedback and game-theoretic approaches.

Conclusions:

• The connection between Backpropagation and dynamic programming offers a new perspective for DNN optimization.
• OCNOpt provides a principled algorithmic approach grounded in optimal control theory.
• This work opens new avenues for designing efficient and robust DNNs.