SimPLE, a visuotactile method learned in simulation to precisely pick, localize, regrasp, and place objects

Area of Science:

• Robotics
• Artificial Intelligence
• Computer Vision

Background:

• Robotic manipulation systems often face a trade-off between task generality and operational precision.
• Current solutions typically involve specialized robots for single tasks, lacking "precise generalization" capabilities.
• Precise pick and place, or kitting, is crucial for transforming unstructured object arrangements into organized ones for further manipulation.

Purpose of the Study:

• To develop a robotic system, SimPLE (Simulation to Pick Localize and placE), capable of precise and general pick and place operations.
• To enable robots to learn pick, regrasp, and place actions for novel objects using only their CAD models.
• To overcome the limitations of existing robotic systems in achieving both generality and precision in manipulation tasks.

Main Methods:

• Task-aware grasping: Computes stable, observable, and placement-favorable grasps.
• Visuotactile perception: Employs supervised learning to match real-world observations with simulated data for accurate object pose estimation.
• Regrasp planning: Solves a shortest-path problem on a hand-to-hand regrasp graph to generate multi-step pick-and-place plans.

Main Results:

• SimPLE successfully performed pick and place operations on 15 diverse objects with varying shapes.
• Achieved over 90% success rate for structured placements with 1-mm clearance for six objects.
• Demonstrated over 80% success rate for 11 objects, showcasing robust performance across a range of items.

Conclusions:

• SimPLE offers a viable solution for precise and general pick and place in robotics.
• The system effectively learns manipulation skills from simulation without prior real-world experience.
• The proposed approach advances robotic manipulation by enabling robots to handle diverse objects with high precision.