Manipulation Planning for Object Re-Orientation Based on Semantic Segmentation Keypoint Detection.

Area of Science:

• Robotics
• Computer Vision
• Artificial Intelligence

Background:

• Robotic manipulation of randomly oriented objects presents a significant challenge.
• Accurate object pose estimation and re-orientation are crucial for automated tasks.
• Existing methods often require precise object models or struggle with variations in object placement.

Purpose of the Study:

• To propose a novel manipulation planning method for robot manipulators to re-orient objects.
• To enable robots to detect and adjust the pose of randomly placed objects to a specified configuration.
• To develop a category-level manipulation strategy independent of specific 3D object models.

Main Methods:

• A 3D keypoint detection system using an RGB-D camera to capture object pose and generate 3D keypoints.
• Semantic segmentation employing Mask Region-Convolutional Neural Network (Mask R-CNN) and Conditional Random Fields (CRFs) for pixel-level object classification.
• Manipulation planning utilizing spherical linear interpolation (Slerp) for generating robot movements based on current and desired object poses.

Main Results:

• The system successfully generates 3D keypoints and suction points for object manipulation.
• Semantic segmentation accurately identifies object parts, enabling pose determination.
• The manipulation planning system effectively re-orients objects to the target pose using Slerp and image-based adjustments.
• Experimental validation using a robot manipulator and vacuum suction cup confirmed the system's capability.

Conclusions:

• The proposed method provides an effective approach for object re-orientation in robotic manipulation.
• Category-level manipulation planning is achievable through semantic segmentation and keypoint detection.
• The system demonstrates robustness in handling randomly placed objects and achieving desired poses.