Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task...

Area of Science:

• Control theory
• Machine learning
• Reinforcement learning

Background:

• Control applications frequently involve tasks with similar, yet distinct, dynamics.
• Existing methods may struggle to efficiently adapt to variations in task parameters.

Purpose of the Study:

• Introduce the Hidden Parameter Markov Decision Process (HiP-MDP) framework.
• Develop a semiparametric regression approach for learning HiP-MDP structure from data.
• Demonstrate the framework's ability to rapidly identify dynamics of new task instances.

Main Methods:

• Parametrization of related dynamical systems using low-dimensional latent factors.
• Semiparametric regression for learning the underlying structure of dynamical systems.
• Empirical evaluation across several task variation settings.

Main Results:

• The learned HiP-MDP successfully models families of related dynamical systems.
• Rapid identification of dynamics for new task instances was achieved.
• The framework demonstrated flexible adaptation to task variations.

Conclusions:

• HiP-MDP offers an effective approach for handling tasks with related dynamics.
• The proposed learning method enables efficient adaptation in control applications.
• This framework advances the ability to generalize control policies across similar tasks.