Performant robotic manipulation with real-world reinforcement learning

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Real-world robotic manipulation requires high reliability and efficiency, often surpassing human capabilities.
• Existing reinforcement learning (RL) methods struggle with robustness and generalization in complex, dynamic environments.

Purpose of the Study:

• To present RL-100, a novel reinforcement learning framework designed for complete task success in real-world robotic manipulation.
• To unify imitation learning and RL for stable, conservative performance improvements.
• To develop a lightweight, efficient control system for high-frequency robotic operations.

Main Methods:

• Developed RL-100, a framework utilizing diffusion visuomotor policies and a unified clipped proximal policy optimization surrogate objective.
• Implemented consistency distillation to compress multistep diffusion models into a one-step controller for low-latency deployment.
• Evaluated the framework on eight diverse real-robot tasks, including manipulation, assembly, and human-interaction scenarios.

Main Results:

• Achieved 100% task success across 1000 evaluated trials (1000 of 1000 episodes) under a predefined protocol.
• Demonstrated zero-shot generalization with ~90% success on unseen tasks and robust adaptation (86.7%) in few-shot scenarios.
• Matched or surpassed expert teleoperators in task completion time and showed high robustness (~96%) to human perturbations.

Conclusions:

• RL-100 offers a viable path towards deployable robot learning by leveraging human priors and aligning training with human-centric metrics.
• The framework demonstrates reliable performance extension beyond human demonstrations, crucial for practical robotic applications.
• The system's adaptability and robustness suggest significant potential for autonomous robotic systems in homes and factories.