Learning Granularity-Aware Affordances From Human-Object Interaction for Tool-Based Functional Dexterous Grasping.

Area of Science:

• Robotics
• Computer Vision
• Human-Robot Interaction

Background:

• Enabling robots to use tools requires precise dexterous gestures for task execution.
• Object affordance features are crucial for agent-object interaction but are underutilized for robotic tool grasping.
• Current methods lack effective ways to leverage affordance cues for functional tool grasping.

Purpose of the Study:

• To propose a granularity-aware affordance feature extraction method for robots.
• To enable robots to locate functional affordance areas and predict dexterous gestures for tool grasping.
• To develop a complete framework for functional tool grasping using learned affordances.

Main Methods:

• Utilized fine-grained affordance features for locating functional object areas.
• Employed coarse-grained affordance features to predict grasp gestures.
• Introduced a weakly supervised approach using exocentric images to supervise egocentric feature extraction.
• Developed a model-based postprocessing module for robotic action execution.

Main Results:

• The proposed GAAF-Dex framework successfully learns granularity-aware affordances from human-object interactions.
• The method outperforms state-of-the-art approaches in localization and gesture prediction tasks.
• Real-world experiments validated the practical applicability of the approach for robotic grasping.

Conclusions:

• The developed method effectively bridges the gap in leveraging affordance cues for robotic tool grasping.
• The weakly supervised approach reduces the need for extensive data annotation.
• The study provides a viable framework and dataset for advancing dexterous robotic manipulation.