Leveraging Human Perception in Robot Grasping and Manipulation Through Crowdsourcing and Gamification

Area of Science:

• Robotics
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Robot grasping in complex environments relies on accurate object attribute recognition.
• Deep learning models struggle with novel object classes and require extensive datasets.
• Human perception and reasoning offer superior capabilities for handling new object categories.

Purpose of the Study:

• To enhance robot grasping capabilities by integrating human intelligence for object recognition and attribute estimation.
• To develop a framework that overcomes limitations of deep learning in recognizing novel objects.
• To create a system that utilizes crowdsourcing and gamification for real-time perception enhancement.

Main Methods:

• A framework combining crowdsourcing and gamification to leverage human intelligence for robot grasping.
• An attribute matching system encoding visual information into an online puzzle game.
• Utilizing collective player intelligence to expand the object attribute database and resolve perception conflicts.

Main Results:

• Demonstrated rapid adaptation to novel object classes using purely visual information and human experience.
• Successfully deployed and evaluated in robotic exoskeleton glove control and autonomous robot grasping.
• Proposed a model for estimating framework response time.

Conclusions:

• The developed framework effectively enhances robot grasping by integrating human intelligence through gamified crowdsourcing.
• This approach provides a scalable solution for improving robot perception in dynamic and unstructured environments.
• The system shows promise for real-time adaptation and handling of previously unseen objects.