Risk-Prioritized Experience Replay for Stable In-Hand Manipulation.

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Deep reinforcement learning (DRL) excels at dexterous in-hand manipulation.
• Current DRL methods often overlook manipulation risk, leading to unstable object handling.
• High-dimensional control in DRL poses challenges for complex manipulation tasks.

Purpose of the Study:

• To introduce a novel replay-sampling strategy, Risk-Prioritized Experience Replay (Risk-PER), for dexterous in-hand manipulation.
• To mitigate overly aggressive behaviors and enhance object handling stability in DRL agents.
• To improve both learning efficiency and manipulation stability by incorporating risk awareness.

Main Methods:

• Developed Risk-Prioritized Experience Replay (Risk-PER) to prioritize low-risk transitions during learning.
• Assigned risk scores to transitions based on manipulation instability indicators.
• Integrated Risk-PER with Deep Deterministic Policy Gradient (DDPG) and evaluated on Allegro Hand manipulation tasks (block and egg) in MuJoCo simulation.

Main Results:

• Risk-PER demonstrated higher success rates compared to baseline methods (HER, reward-penalty).
• The proposed method achieved lower manipulation risk and more stable learning behavior.
• Risk-PER effectively biased replay toward lower-risk experiences while enabling learning from risk-related events.

Conclusions:

• Incorporating task-specific risk awareness into replay prioritization significantly improves learning efficiency.
• Risk-PER enhances manipulation stability in dexterous in-hand tasks.
• The proposed strategy offers a promising direction for developing safer and more robust robotic manipulation systems.