Computer Vision-Based Grasp Pattern Recognition With Application to Myoelectric Control of Dexterous Hand Prosthesis

Area of Science:

• Robotics and Artificial Intelligence
• Biomedical Engineering
• Computer Vision

Background:

• Dexterous prostheses require sophisticated control for effective object manipulation.
• Current myoelectric control methods can be demanding and limited in versatility.
• AI offers new possibilities for autonomous prosthetic control.

Purpose of the Study:

• To develop a computer vision-based classification method for object grasp patterns.
• To enable autonomous control of multi-fingered prosthetic hands for daily tasks.
• To reduce the demand on myoelectric control for prosthetic users.

Main Methods:

• An RGB-D image database of 121 objects was created, categorized into four grasp patterns (cylindrical, spherical, tripod, lateral).
• Data included variations in object size, shape, posture, illumination, and camera position.
• A multilayer Convolutional Neural Network (CNN) was trained and cross-validated using different input data (RGB, Grayscale-Depth).

Main Results:

• Depth data significantly improved grasp pattern recognition accuracy compared to RGB alone.
• Integrating grayscale and depth (Gray-D) data enhanced classification accuracy by over 10% compared to RGB.
• The CNN achieved 98% accuracy on the database and 93.9% on novel samples, demonstrating strong generalization.
• The Vision-EMG system improved task completion time by 6.4 seconds compared to Coding-EMG (13 seconds).

Conclusions:

• A novel computer vision approach effectively classifies objects for prosthetic hand grasps.
• Depth information is crucial for accurate grasp pattern recognition in prosthetic control.
• The Vision-EMG method significantly enhances the efficiency and usability of dexterous prosthetic hands.