Reward-weighted regression with sample reuse for direct policy search in reinforcement learning

Area of Science:

• Robotics
• Machine Learning
• Reinforcement Learning

Background:

• Direct policy search is effective for continuous, high-dimensional systems.
• High sampling costs limit the applicability of policy search methods.
• Efficient sample reuse is crucial for reducing computational burden.

Purpose of the Study:

• To extend expectation-maximization-based policy search for efficient sample reuse.
• To introduce a novel method, reward-weighted regression with sample reuse (R3).
• To demonstrate the effectiveness of R3 in practical applications.

Main Methods:

• Extension of expectation-maximization-based policy search.
• Implementation of reward-weighted regression with sample reuse (R3).
• Validation through robot learning experiments.

Main Results:

• The proposed R3 method enables efficient reuse of previously collected samples.
• Reduced sample requirements for stable policy updates.
• Successful demonstration of R3 in robot learning tasks.

Conclusions:

• R3 significantly improves the sample efficiency of direct policy search.
• The method is effective for controlling continuous, high-dimensional systems.
• R3 offers a practical solution for expensive sampling scenarios in reinforcement learning.