Reinforcement learning for decision-making under deep uncertainty.

Area of Science:

• Decision Sciences
• Artificial Intelligence
• Computational Science

Background:

• Planning under complex uncertainty requires adaptive strategies.
• Current exploration methods for policy performance assessment lack inherent adaptiveness, reducing decision-making efficiency.
• Automated adaptive policy-making is crucial for effective planning.

Purpose of the Study:

• Introduce Reinforcement Learning (RL) with closed-loop control as a novel exploration method for planning under uncertainty.
• Compare the performance of RL against the Multi-Objective Evolutionary Algorithm (MOEA).
• Evaluate the efficiency and robustness of RL and MOEA in handling different types of uncertainty.

Main Methods:

• Computational experiments comparing RL and MOEA on two hypothetical problems.
• Utilizing Reinforcement Learning (RL) with closed-loop control for adaptive policy generation.
• Employing Multi-Objective Evolutionary Algorithm (MOEA) as a benchmark for exploration.

Main Results:

• Reinforcement Learning (RL) demonstrates higher efficiency and policy robustness against parameter uncertainty by leveraging exploration history.
• Multi-Objective Evolutionary Algorithm (MOEA) offers more intuitive quantification of objective uncertainty, leading to better robustness in that domain.
• RL and MOEA exhibit complementary strengths in addressing different facets of uncertainty in planning.

Conclusions:

• Reinforcement Learning (RL) provides an effective approach for automated adaptive policy-making in uncertain environments.
• The choice between RL and MOEA depends on the specific type of uncertainty (parameter vs. objective) prevalent in the planning problem.
• Findings guide researchers in selecting optimal exploration methods for planning under complex uncertainty.