Flexible robotic hand harnesses large deformations for full-coverage human-like multimodal haptic perception.

Area of Science:

• Robotics
• Sensory Systems
• Human-Robot Interaction

Background:

• Integrating human-level haptic perception into soft grippers is crucial for safe robotic grasping and interaction.
• Existing visual-tactile sensors often compromise compliance for optical stability, limiting non-planar contact perception.

Purpose of the Study:

• To develop a compliant soft gripper with integrated visual-tactile sensing for enhanced perception and interaction.
• To replicate multiple human tactile modalities using a single-camera system.

Main Methods:

• The FlexiRay soft gripper utilizes the Fin Ray Effect, a multi-layered flexible substrate, an optimized multi-mirror optical system, and a decoupled deep learning framework.
• The system integrates a single camera to capture visual-tactile data.

Main Results:

• FlexiRay replicates five human tactile modalities: force, contact location, texture, temperature, and proprioception.
• Achieved high accuracy: 0.17 N force, 0.96 mm spatial resolution, 0.24 mm proprioception, and 1.17°C temperature accuracy.
• Maintained over 90% effective sensory coverage with high compliance.

Conclusions:

• FlexiRay offers a compliant, multimodal soft gripper with human-level haptic perception capabilities.
• The gripper enhances safe collaboration, intelligent teleoperation, and service robotics.
• This technology advances robotic intelligence, safety, and real-world applicability.