Recognition of Grasping Patterns Using Deep Learning for Human-Robot Collaboration

Area of Science:

• Robotics
• Computer Vision
• Artificial Intelligence

Background:

• Collaborative robots require prediction and anticipation abilities for shared tasks.
• Accurate object recognition by robots is crucial for understanding human operator intentions.

Purpose of the Study:

• To develop a learning-based framework for assistive robots to recognize grasped objects using hand and finger joint patterns.
• To compare the performance of Convolutional Neural Networks (CNNs) and Transformers for object recognition.

Main Methods:

• Utilized MediaPipe for detecting hand landmarks from RGB images.
• Developed a deep multi-class classifier to predict objects from extracted keypoints.
• Evaluated CNN and Transformer architectures based on accuracy, precision, recall, and F1-score.

Main Results:

• The proposed framework demonstrated effective object recognition capabilities.
• Both CNN and Transformer models showed varying performance metrics.
• Insights were gained into factors affecting model generalization.

Conclusions:

• The developed framework successfully enables assistive robots to recognize objects manipulated by humans.
• The study provides a comparative analysis of deep learning architectures for this task.
• Findings highlight the importance of generalization in human-robot interaction systems.