Efficient Bayesian Policy Reuse With a Scalable Observation Model in Deep Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Bayesian policy reuse (BPR) is a framework for policy transfer using task belief inference.
• Existing BPR methods often rely on limited, end-of-episode signals like episodic return.
• Current BPR approaches can be sample-inefficient and struggle with scalable observation models.

Purpose of the Study:

• To enhance Bayesian policy reuse (BPR) for more efficient policy transfer in deep reinforcement learning (DRL).
• To address limitations in observation signals and the scalability of observation models in traditional BPR.
• To adapt BPR for continual learning scenarios, preventing negative transfer.

Main Methods:

• Replaced episodic return with informative, instantaneous state transition samples for task inference.
• Developed a scalable observation model by fitting state transition functions from limited samples.
• Extended the framework to continual learning using a plug-and-play scalable observation model.

Main Results:

• The proposed method achieves faster and more accurate task inference compared to traditional BPR.
• The scalable observation model effectively generalizes across tasks with minimal samples.
• The continual learning extension successfully mitigates negative transfer in new, unknown tasks.

Conclusions:

• The improved BPR method significantly enhances policy transfer efficiency in DRL.
• The novel observation signal and scalable model offer practical advantages for real-world applications.
• The adaptation for continual learning broadens the applicability of BPR to dynamic environments.