Dataset with Tactile and Kinesthetic Information from a Human Forearm and Its Application to Deep Learning

Area of Science:

• Robotics
• Human-Robot Interaction
• Machine Learning

Background:

• Physical Human-Robot Interaction (pHRI) requires precise grasping of human limbs for safety in applications like assistive robotics.
• Existing methods like computer vision have limitations in unstructured environments.
• Tactile and proprioceptive data from grippers offer valuable insights into human-robot contact and body features.

Purpose of the Study:

• To present a novel dataset of tactile and kinesthetic data from a robot gripper grasping a human forearm.
• To develop and evaluate a fusion approach for estimating the grasped forearm section using this data.
• To demonstrate the effectiveness of deep learning models in analyzing sequential sensor data for improved grasping estimation.

Main Methods:

• Collected a dataset using a three-fingered robot gripper with tactile sensors on a human forearm.
• Employed a palpation procedure to map forearm shape, bones, and muscles.
• Utilized Long Short-Term Memory (LSTM) neural networks to process sequential tactile and kinesthetic data separately.
• Implemented a fusion neural network to combine outputs from individual LSTMs for enhanced estimation.

Main Results:

• Individual LSTM models showed good performance in training with tactile and kinesthetic data.
• The fusion approach significantly improved the accuracy of estimating the grasped forearm section compared to separate models.
• The developed deep learning models effectively leveraged sequential sensor information.

Conclusions:

• The novel dataset provides valuable resources for advancing pHRI grasping research.
• Fusion of tactile and kinesthetic data with deep learning offers a robust method for precise forearm section estimation.
• This approach enhances safety and performance in robotic applications involving human contact.