Robust and Efficient Transfer Learning with Hidden Parameter Markov Decision Processes.

Area of Science:

• Machine Learning
• Reinforcement Learning
• Probabilistic Modeling

Background:

• Markov Decision Processes (MDPs) are foundational for sequential decision-making.
• Modeling families of related tasks efficiently remains a challenge.
• Existing frameworks may struggle with joint uncertainty and scalability.

Purpose of the Study:

• To introduce a novel formulation of the Hidden Parameter Markov Decision Process (HiP-MDP).
• To enhance the modeling of joint uncertainty in latent parameters and state spaces.
• To improve scalability for complex, high-dimensional reinforcement learning problems.

Main Methods:

• Developed a new HiP-MDP formulation incorporating joint uncertainty.
• Replaced Gaussian Process models with Bayesian Neural Networks for inference.
• Applied the enhanced framework to problems with higher dimensions and complex dynamics.

Main Results:

• The new HiP-MDP formulation accurately models joint uncertainty.
• Bayesian Neural Networks enable more scalable inference compared to previous methods.
• The expanded scope allows application to more complex and higher-dimensional tasks.

Conclusions:

• The enhanced HiP-MDP offers a more robust and scalable approach for multi-task learning.
• This framework advances the ability to model complex dynamics and uncertainties in related tasks.
• The use of Bayesian Neural Networks broadens the applicability of HiP-MDPs.