Generating Realistic Sound with Prosthetic Hand: A Reinforcement Learning Approach

Area of Science:

• Robotics
• Bioacoustics
• Machine Learning

Background:

• Auditory feedback is crucial for material identification, enhancing prosthetic hand functionality.
• Current prosthetic hands lack the ability to accurately reproduce sounds for tactile-acoustic feedback.
• Distinguishing materials via sound (e.g., drywall vs. brick) is a significant challenge for prosthetic users.

Purpose of the Study:

• To enable prosthetic hands to accurately reproduce sounds for material discrimination.
• To enhance prosthetic device functionality and user experience through auditory feedback.
• To develop a method for generating human-like sound characteristics in prosthetic hands.

Main Methods:

• Utilized reinforcement learning (RL) techniques to train prosthetic hands.
• Focused on emulating human-like sound characteristics, specifically amplitude and onset timing.
• Developed a tailored reward function based on amplitude, onset strength, and timing criteria.

Main Results:

• Prosthetic hands were trained to generate sounds mimicking human-like auditory signals.
• The approach integrated sound attribute analysis to guide prosthetic hand movements.
• The reward function ensured close alignment between prosthetic movements and desired sound output.

Conclusions:

• Reinforcement learning can train prosthetic hands to produce accurate auditory feedback.
• This technology has the potential to significantly improve material identification for prosthetic users.
• Mimicking human-like sound characteristics is key to advancing prosthetic sensory capabilities.