Behavior fusion for deep reinforcement learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Designing reward functions for complex tasks in deep reinforcement learning (DRL) is challenging.
• Existing methods often require extensive task-specific reward engineering and training.
• This necessitates novel approaches for efficient policy learning in DRL.

Purpose of the Study:

• To propose a behavior fusion framework for actor-critic architectures to address reward function design challenges in DRL.
• To enable learning policies for complex tasks by decomposing them into simpler sub-tasks.
• To facilitate rapid prototyping and policy assembly for intricate DRL applications.

Main Methods:

• Decomposition of complex tasks into multiple sub-tasks, each trained individually with a simple reward function.
• Merging pre-trained sub-task policies into a unified policy for the overall complex task.
• Integration of modules in policy gradient calculation using accumulated returns to minimize variance.

Main Results:

• The behavior fusion framework successfully decomposes and merges policies for complex tasks.
• The proposed method demonstrates comparable or improved performance against a gate network approach in validation tasks.
• Effective policy learning was achieved without designing a new reward function for the complex task.

Conclusions:

• Behavior fusion offers an effective strategy for tackling complex tasks in deep reinforcement learning by leveraging modularity.
• This approach simplifies the reward function design process and enables faster development of DRL agents.
• The framework provides a robust method for building complex task policies from pre-trained sub-task components.