Cross-Modal Reconstruction for Tactile Signal in Human-Robot Interaction

Area of Science:

• Robotics
• Computer Vision
• Human-Robot Interaction (HRI)

Background:

• Humans naturally infer interaction forces from visual cues due to prior experience in human-robot interaction (HRI).
• Accurate tactile sensing is crucial for safe and effective HRI, but direct tactile sensing can be complex or limited.
• Cross-modal signal processing offers a potential solution for inferring tactile information from other sensory inputs.

Purpose of the Study:

• To propose and validate a novel method for reconstructing tactile information using visual data.
• To enable the inference of tactile interaction forces solely from visual information.
• To reduce network complexity and improve material identification accuracy in HRI.

Main Methods:

• Processing of Groups of Pictures (GOPs) as input data.
• Utilizing a low-rank foreground-based attention mechanism (LAM) to identify Regions of Interest (ROIs).
• Employing a linear regression convolutional neural network (LRCNN) for contact force inference from video frames.

Main Results:

• Experimental validation confirms the feasibility of cross-modal reconstruction for tactile information.
• The proposed method successfully infers contact forces from visual input.
• Demonstrated reduction in network complexity compared to existing approaches.
• Improved material identification accuracy was observed.

Conclusions:

• Cross-modal signal processing, integrating visual and tactile information, is a feasible approach for inferring interaction forces in HRI.
• The developed LRCNN model effectively infers contact forces using visual data, offering a simplified yet accurate solution.
• This method holds promise for enhancing robotic perception and interaction capabilities by leveraging visual cues for tactile sensing.