A Multimodal Intention Detection Sensor Suite for Shared Autonomy of Upper-Limb Robotic Prostheses.

Area of Science:

• Neurorobotics and Human-Machine Interfaces
• Biomedical Engineering
• Rehabilitation Technology

Background:

• Neurorobotic augmentation aids individuals with impaired motor functions but faces challenges with high cognitive load for human-machine interfaces (HMI).
• Current HMI systems often use limited control strategies, restricting simultaneous multi-degree-of-freedom (DoF) control, which is crucial for complex tasks like grasping.

Purpose of the Study:

• To develop and evaluate a shared autonomy framework using a low-cost multimodal sensor suite to predict user grasp intent and automate robotic hand control.
• To reduce the cognitive burden associated with controlling neurorobotic devices for individuals with motor impairments.

Main Methods:

• A multimodal sensor suite was developed, fusing mechanomyography (MMG), camera-based visual information, and inertial measurement.
• The system extracted 84 motion features to predict grasp intent based on dynamical features during natural motions.
• Tests were conducted with able-bodied and amputee participants grasping common household objects using a robotic hand.

Main Results:

• Real-time grasp classification achieved high accuracy: 100% for bottles, 82.5% for lids, and 88.9% for boxes.
• The system successfully predicted user intent for various grasping actions.
• The multimodal approach demonstrated effective prediction of different grasp strategies and automation of task performance.

Conclusions:

• The proposed multimodal sensor suite offers a novel approach for predicting grasp strategies and automating tasks in neurorobotic systems.
• This framework has the potential to enhance the usability of upper-limb prosthetic devices and modern neurorobotic systems through intuitive control.
• The system effectively addresses the challenge of cognitive load in human-machine interaction for motor-impaired individuals.