Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device.

Area of Science:

• Human-computer interaction
• Robotics
• Mixed Reality

Background:

• Mixed reality (MR) offers intuitive visual experiences and comfortable human-robot interaction.
• Wearable robotic limbs (supernumerary robotic limbs, SRL) can augment human capabilities.
• Effective communication is crucial for seamless human-robot collaboration.

Purpose of the Study:

• To apply a mixed reality device (Hololens2) for interactive communication between a wearer and a wearable robotic limb.
• To design task models and interaction schemes for integrating Hololens2 with a wearable robotic arm.
• To evaluate the effectiveness of human augmentation in practical tasks.

Main Methods:

• Utilized Hololens2 to capture wearer's data (eye gaze, hand gestures, voice).
• Implemented Hololens2 for augmented reality and audio feedback to the wearer.
• Developed task models and interaction schemes for cable installation and electrical connector soldering.
• Integrated a wearable robotic arm with Hololens2 for augmented capabilities.

Main Results:

• Demonstrated the feasibility of using Hololens2 as a communication bridge in human-robot interaction.
• Designed practical application scenarios in aircraft manufacturing.
• Evaluated human augmentation through task completion time statistics.

Conclusions:

• Mixed reality devices like Hololens2 can effectively enhance human-robot interaction with wearable robotic limbs.
• The proposed system shows potential for augmenting human capabilities in complex manufacturing tasks.
• Further evaluation of task completion times indicates improved efficiency through this integrated approach.