Robotic Pick-and-Place With Uncertain Object Instance Segmentation and Shape Completion

Area of Science:

• Robotics
• Computer Vision
• Artificial Intelligence

Background:

• Robotic manipulation of novel objects with partial visibility presents challenges in precise placement.
• Accurately modeling object geometry and accounting for perceptual uncertainty are critical for successful grasping and packing.

Purpose of the Study:

• To develop and evaluate a cost function for robotic regrasp planning that incorporates perceptual uncertainty.
• To improve the success rate of tightly packing partially visible, novel objects into a bin.

Main Methods:

• Object instance segmentation and shape completion were used to model objects.
• A regrasp planner incorporated perceptual uncertainty into its cost function.
• Seven different cost functions were compared, including one using neural networks for grasp and place stability estimation.

Main Results:

• The proposed cost function, utilizing neural networks, significantly outperformed uncertainty-unaware costs.
• The new method achieved a 7.8% higher success rate in tightly packing objects compared to the minimum-number-of-grasps cost.
• The neural network-based cost evaluated faster than Monte Carlo sampling.

Conclusions:

• Incorporating perceptual uncertainty into the regrasp planner's cost function is crucial for effective robotic manipulation.
• The developed cost function enhances the performance of robots in complex pick-and-place tasks, particularly for dense packing scenarios.