Approximate neural optimal control with reinforcement learning for a torsional pendulum device.

Area of Science:

• Control Engineering
• Nonlinear Systems
• Machine Learning

Background:

• Nonaffine nonlinear systems present significant challenges for optimal feedback control design.
• Torsional pendulum devices with hyperbolic tangent nonlinearities serve as a relevant model for nonaffine systems.

Purpose of the Study:

• To investigate an approximate optimal control design for continuous-time nonaffine nonlinear systems.
• To develop and validate a reinforcement learning-based control strategy for complex plants.

Main Methods:

• System transformation using a pre-compensation technique to facilitate the learning algorithm.
• Integral policy iteration strategy to reduce reliance on explicit system dynamics.
• Actor-critic architecture implemented with neural network approximators.

Main Results:

• Successful experimental verification of the proposed control design on a torsional pendulum plant.
• Demonstrated stability performance with a basic robustness guarantee after 20 learning iterations.
• Effective handling of nonaffine nonlinearities through the developed reinforcement learning approach.

Conclusions:

• The proposed reinforcement learning framework offers a viable solution for approximate optimal control of nonaffine nonlinear systems.
• The integral policy iteration and actor-critic methods effectively address the complexities of these systems.
• Experimental results confirm the stability and robustness of the control strategy for the torsional pendulum.