The Helping Hand: An Assistive Manipulation Framework Using Augmented Reality and Tongue-Drive Interfaces

Area of Science:

• Robotics
• Human-Computer Interaction
• Assistive Technology

Background:

• Physical disabilities often necessitate assistive systems for daily tasks.
• Reducing user cognitive burden is key for effective human-robot collaboration.
• Existing control methods can be complex for users with limited mobility.

Purpose of the Study:

• To develop a human-in-the-loop system for collaborative manipulation tasks for individuals with physical disabilities.
• To decrease the cognitive burden on users by increasing system autonomy.
• To evaluate the system's performance against traditional control methods.

Main Methods:

• A framework was developed to interpret high-level user intent for manipulation tasks.
• Sensor input processed environmental data, with augmented reality glasses providing visual feedback and robot affordances.
• A tongue drive system served as the primary input modality for controlling a robotic arm.

Main Results:

• The proposed system demonstrated competitive performance compared to Cartesian control and state-of-the-art approaches.
• Task completion times were reduced, indicating increased efficiency.
• User cognitive burden was observed to decrease with enhanced system autonomy.

Conclusions:

• The human-in-the-loop system effectively supports collaborative manipulation for people with physical disabilities.
• The integration of augmented reality and tongue drive technology simplifies complex tasks.
• This approach offers a promising direction for enhancing independence and quality of life.