Continuous Value Assignment: A Doubly Robust Data Augmentation for Off-Policy Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Deep reinforcement learning (RL) excels in control tasks but suffers from sample inefficiency.
• Existing data augmentation methods struggle with high-dimensional, redundant features in environment dynamics.
• Modeling environment transitions accurately is challenging for complex RL tasks.

Purpose of the Study:

• To introduce Continuous Value Assignment (CVA), an optimization-level data augmentation technique.
• To address the sample inefficiency of deep reinforcement learning without explicit transition modeling.
• To enhance the training of RL agents in complex, continuous control tasks.

Main Methods:

• CVA synthesizes novel training data directly in the state-action value space.
• It combines parameterized value prediction with nonparametric value interpolation.
• This approach generates doubly robust target values for novel states and actions.

Main Results:

• CVA significantly improves sample efficiency in complex continuous control tasks.
• The method surpasses the performance of several advanced reinforcement learning baselines.
• Experiments demonstrate the effectiveness of CVA in diverse control scenarios.

Conclusions:

• Continuous Value Assignment offers a novel and effective approach to data augmentation in RL.
• By directly targeting state-action values, CVA bypasses the limitations of transition modeling.
• The proposed method shows strong potential for advancing sample-efficient reinforcement learning.