Safety-Guaranteed, Accelerated Learning in MDPs with Local Side Information.

Area of Science:

• Robotics and Control Systems
• Artificial Intelligence and Machine Learning

Background:

• Optimal control policy synthesis in uncertain environments requires extensive exploration.
• Classical learning algorithms are often limited by the time steps needed for environment model learning.
• Real-world applications necessitate efficient learning and guaranteed agent safety.

Purpose of the Study:

• To develop an algorithm for accelerated learning of environment models in uncertain dynamics.
• To balance exploration and exploitation using local side information for faster policy synthesis.
• To define and guarantee agent safety during the exploration process.

Main Methods:

• Generalization of indirect sampling for accelerated learning by incorporating local side information.
• Formalization of the value of information in Markov decision processes to guide exploration.
• Maximization of the estimated value of learned information at each time step.

Main Results:

• The proposed algorithm significantly accelerates the learning process compared to classical methods.
• Guarantees on agent safety are provided under specific assumptions for exploration.
• Numerical experiments with a Mars rover demonstrate improved learning speed and safety.

Conclusions:

• Exploiting local side information effectively accelerates learning and optimal control policy synthesis.
• The value of information framework provides a principled way to guide exploration.
• The algorithm ensures agent safety, crucial for physical systems like autonomous rovers.