IMPLICIT DUAL CONTROL BASED ON PARTICLE FILTERING AND FORWARD DYNAMIC PROGRAMMING.

Area of Science:

• Control Theory
• Stochastic Systems
• Machine Learning

Background:

• Implicit dual control synthesizes stochastic control policies by approximating Bellman's dynamic programming equations.
• Explicit dual control methods artificially induce probing by modifying cost functions to reward learning.
• Existing methods often lack systematic approximation of underlying stochastic dynamics.

Purpose of the Study:

• To develop a novel sampling-based approach for implicit dual control.
• To integrate particle filtering with policy iteration for forward dynamic programming.
• To provide a computationally efficient method for real-time applications.

Main Methods:

• Combines a particle filter with a policy-iteration method for forward dynamic programming.
• Utilizes a specific H-block architecture for implementation.
• Applies practical suggestions for reducing computational loads within the H-block.

Main Results:

• The integrated approach provides a complete sampling-based solution to implicit dual control.
• The H-block architecture simplifies implementation and allows for real-time applications.
• Simulations on a stochastic pendulum model with unknown parameters demonstrate improved closed-loop performance.

Conclusions:

• The proposed sampling-based implicit dual control method systematically improves performance over common stochastic control approaches.
• The integration of particle filters and policy iteration offers a robust framework.
• The H-block architecture facilitates practical, real-time implementation of advanced control strategies.