Neuro-Evolutionary Direct Policy Search for Multiobjective Optimal Control

Area of Science:

• Artificial Intelligence
• Control Theory
• Operations Research

Background:

• Direct policy search (DPS) is a key reinforcement learning (RL) method for multiobjective Markov decision processes (MOMDPs).
• Traditional DPS methods select a fixed policy functional class a priori, which can limit performance across different objective tradeoffs.
• The selection of the policy class is critical as it defines the search space for optimal control policies.

Purpose of the Study:

• To introduce neuro-evolutionary multiobjective direct policy search (NEMODPS), a novel routine that jointly searches policy architecture and parameters.
• To address the limitation of fixed policy class selection in traditional DPS for MOMDPs.
• To develop a tradeoff-dynamic approach for policy search in complex control problems.

Main Methods:

• NEMODPS employs an evolutionary strategy, starting with simple neural networks.
• It progressively enhances network architectures and parameters through mutation and crossover operations.
• Selection is based on performance across multiple objectives, optimizing both structure and coefficients.

Main Results:

• NEMODPS demonstrated consistent performance across multiple runs in designing control policies for a water system.
• The method effectively searches policy structures and parameters dynamically based on objective tradeoffs.
• NEMODPS significantly outperformed traditional DPS methods that use predefined policy topologies.

Conclusions:

• NEMODPS offers a more effective and adaptive approach to designing optimal control policies for MOMDPs.
• The joint search of policy architecture and parameters overcomes limitations of fixed functional class selection.
• This neuro-evolutionary method provides a robust and superior alternative for complex control applications.