An Inexact Sequential Quadratic Programming Method for Learning and Control of Recurrent Neural Networks

Area of Science:

• Artificial Intelligence
• Machine Learning
• Control Theory

Background:

• Partially Observable Markov Decision Processes (POMDPs) present significant challenges in modeling and control due to hidden states.
• Existing methods often struggle with scalability and computational efficiency for complex systems.

Purpose of the Study:

• To develop an effective two-stage approach for solving POMDPs.
• To introduce a novel framework for the identification and control stages of POMDPs.

Main Methods:

• An inexact sequential quadratic programming framework for recurrent neural network learning (iSQPRL) was employed for the identification stage, approximating the system with a dynamically consistent overshooting recurrent neural network (DCRNN).
• The learning problem was formulated as a constrained optimization problem, with analysis of the quadratic programming (QP) subproblem using a restarted Krylov-subspace iterative scheme.
• A generalized Gauss-Newton (GGN) algorithm was adapted for the control stage, designing a feedforward neural network (FNN) controller atop the RNN model.

Main Results:

• The iSQPRL framework demonstrated effectiveness in approximating the system dynamics within the POMDP identification stage.
• The GGN algorithm facilitated efficient design of the FNN controller, leveraging approximations of curvature terms and a regularization function property for mini-batch step size selection.
• Simulation results validated the overall approach, showing its efficacy in solving POMDPs.

Conclusions:

• The proposed two-stage approach offers a robust and effective solution for POMDPs.
• The integration of iSQPRL for identification and GGN for control represents a significant advancement in handling complex sequential decision-making problems under partial observability.