Assisting Upper Limb Prosthesis with a Computer Vision System for Material Detection

The concept of utilizing computer vision to aid in the control of prosthetic limbs has been explored in recent studies. However, many computer vision systems are challenged by noisy and cluttered backgrounds in realistic scenarios. In this study, we investigated the feasibility of using a state-of-the-art object detection model (YOLOv8) to identify objects grasped by a myoelectric prosthetic hand. A custom model was trained to recognize various types of cups and was integrated with an EMG-based prosthesis controller. The classification accuracy of the model during an object grasping task with a human subject was approximately 93%. Predictions of grasped objects will be used in future work to adjust the parameters of a grip force controller for a prosthetic hand based on the detected object's properties.