Dichotomy value iteration with parallel learning design towards discrete-time zero-sum games.

Area of Science:

• Control Systems Engineering
• Game Theory
• Machine Learning

Background:

• Zero-sum games are critical in decision-making for nonlinear systems.
• Existing value iteration (VI) algorithms can be slow to converge.
• Parallel computing offers potential for accelerating learning processes.

Purpose of the Study:

• To develop a novel parallel learning framework for discrete-time nonlinear zero-sum games.
• To establish a tentative function to guide the learning process of parallel controllers.
• To accelerate the learning speed of optimal policy determination.

Main Methods:

• Development of a parallel learning framework utilizing a tentative function derived from prior knowledge of value iteration (VI).
• Implementation of two exploration policies to compress the optimal cost function's neighborhood.
• Establishment of a novel dichotomy VI algorithm based on the parallel learning framework.

Main Results:

• The parallel learning framework effectively guides the learning process of controllers.
• The dichotomy VI algorithm significantly accelerates the learning speed.
• Demonstrated convergence of parallel controllers to the optimal policy from diverse initial policies.

Conclusions:

• The proposed parallel learning framework and dichotomy VI algorithm are effective for solving discrete-time nonlinear zero-sum games.
• The novel approach enhances learning speed and ensures convergence to optimal policies.
• Validated through simulations on two typical nonlinear systems.