Multi-Grasp Classification for the Control of Robot Hands Employing Transformers and Lightmyography Signals

Area of Science:

• Biomedical Engineering
• Human-Computer Interaction
• Robotics

Background:

• Smart devices require intuitive muscle-machine interfaces (MuMI) for immersive interaction.
• Electromyography (EMG) is common but has drawbacks, prompting research into alternatives.
• Lightmyography (LMG) was previously introduced as a promising MuMI technology.

Purpose of the Study:

• To experimentally validate the efficiency of the LMG armband for classifying a large set of gestures.
• To evaluate the performance of a deep learning model, Temporal Multi-Channel Vision Transformers (TMC-ViT), with LMG data.
• To assess the real-time control capabilities of the LMG interface for robotic applications.

Main Methods:

• LMG armband data from six participants performing thirty-two distinct gestures were collected.
• A Temporal Multi-Channel Vision Transformers (TMC-ViT) deep learning model was employed for gesture classification.
• Two undersampling techniques were compared to optimize classifier performance.
• The LMG interface was used for real-time control of a robotic hand executing ten gestures.

Main Results:

• The LMG interface achieved high accuracy, reaching up to 92% for thirty-two gesture classification.
• The TMC-ViT model demonstrated significant efficiency in processing LMG signals.
• Real-time robotic hand control was successfully demonstrated using ten distinct gestures, replicating various grasp types.

Conclusions:

• Lightmyography (LMG) is a highly efficient technology for advanced muscle-machine interfaces.
• The study validates LMG's potential for complex gesture recognition and intuitive human-robot interaction.
• LMG offers a viable alternative to EMG for controlling robotic systems in real-time.