Continual deep reinforcement learning with task-agnostic policy distillation.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Continual learning systems aim to solve multiple tasks without retraining.
• Catastrophic forgetting, lack of positive transfer, scalability, and unlabeled data are key challenges.
• Current methods require significant training time for each new task.

Purpose of the Study:

• To introduce a novel framework, Task-Agnostic Policy Distillation (TAPD), to address limitations in continual learning.
• To enable agents to learn new tasks efficiently without forgetting previously acquired knowledge.
• To improve sample efficiency and scalability in universal learning systems.

Main Methods:

• The proposed Task-Agnostic Policy Distillation (TAPD) framework incorporates a task-agnostic exploration phase.
• Agents maximize intrinsic motivation during exploration, seeking novel states in a self-supervised manner.
• Knowledge gained in the task-agnostic phase is distilled for efficient downstream task learning.

Main Results:

• The TAPD framework effectively alleviates catastrophic forgetting and enhances positive forward transfer.
• The approach demonstrates improved scalability across numerous tasks.
• Agents trained with TAPD exhibit significantly improved sample efficiency in solving downstream tasks.

Conclusions:

• Task-Agnostic Policy Distillation (TAPD) provides an effective solution for continual learning challenges.
• The self-supervised, task-agnostic exploration enables efficient knowledge transfer and adaptation.
• TAPD advances the development of more robust and scalable universal learning systems.