Lifelong Incremental Reinforcement Learning With Online Bayesian Inference.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Long-lived reinforcement learning (RL) agents require continuous adaptation to dynamic environments.
• Incremental learning is crucial for agents to build upon past experiences and adapt efficiently.

Purpose of the Study:

• To propose lifelong incremental reinforcement learning (LLIRL), an algorithm for efficient lifelong adaptation in dynamic environments.
• To develop a method for agents to incrementally adapt their behavior and learning processes.

Main Methods:

• Developed a library of infinite mixture of parameterized environment models using a Chinese restaurant process (CRP) prior.
• Employed the expectation-maximization (EM) algorithm with online Bayesian inference for incremental updates.
• Utilized E-step for posterior expectation estimation and M-step for parameter updates.

Main Results:

• The LLIRL algorithm effectively clusters environment parameters in a latent space.
• New environment models are instantiated automatically, and old models are retrieved when environments reappear.
• Simulation experiments showed LLIRL outperforms existing methods in dynamic environments.

Conclusions:

• LLIRL enables effective incremental adaptation to dynamic environments for lifelong learning.
• The algorithm facilitates continuous adaptation and efficient learning by managing a dynamic set of environment models.