Dexterous Manipulation Based on Object Recognition and Accurate Pose Estimation Using RGB-D Data

Area of Science:

• Robotics
• Computer Vision
• Industrial Automation

Background:

• Automating industrial valve manipulation requires precise object recognition and pose estimation.
• Existing systems face challenges with varying object orientations and distances.

Purpose of the Study:

• To develop an integrated system for object recognition, six-degrees-of-freedom pose estimation, and dexterous manipulation.
• To enhance the accuracy of pose estimation for industrial valves using multi-perspective point clouds.
• To create a robust manipulation strategy for challenging scenarios.

Main Methods:

• An Intel RealSense D435 camera and JACO robotic arm were utilized.
• Object recognition involved scene segmentation, geometric and model recognition, and dynamic cluster merging.
• Pose estimation employed the random sample consensus algorithm.
• A zone-based dexterous manipulation strategy was developed to adjust camera positioning.

Main Results:

• The system demonstrated reliable performance within acceptable error thresholds for objects within a ±15° view range.
• Increased errors were observed at extreme orientations and distances, particularly for ball valves.
• The zone-based manipulation strategy effectively mitigated errors in difficult scenarios, improving reliability.

Conclusions:

• The integrated system offers improved object recognition and pose estimation for robotic manipulation.
• The zone-based strategy enhances dexterous manipulation reliability in industrial settings.
• This research contributes a novel robot motion model for industrial automation.