A multi-scale robotic tool grasping method for robot state segmentation masks

Area of Science:

• Robotics
• Artificial Intelligence
• Computer Vision

Background:

• Robots increasingly collaborate with humans in shared workspaces.
• Effective human-robot collaboration requires robots to understand tool usage and task context.

Purpose of the Study:

• To develop a modular system for robots to better understand and participate in collaborative tasks involving tools.
• To improve the accuracy and efficiency of robot grasp and handover predictions in human-robot interactions.

Main Methods:

• A multi-layer instance segmentation network identifies task-related tools and classifies objects based on robot state.
• A multi-scale grasping network (MGR-Net) predicts optimal grasp and handover configurations using state semantic region masks.
• A novel real-world tool dataset was constructed for system evaluation.

Main Results:

• The system accurately identifies tools and generates state semantic regions, classifying robot roles as 'leader' or 'assistant'.
• MGR-Net demonstrated higher accuracy in predicting grasp and handover configurations compared to traditional methods.
• The system achieved strong performance on untrained real-world tool datasets and was validated on a Sawyer robot platform.

Conclusions:

• The proposed modular system enhances robot understanding of tool-use in collaborative tasks.
• The MGR-Net effectively predicts grasp and handover configurations, improving human-robot interaction.
• The system shows significant potential for real-world applications in collaborative robotics.