An 18-DOF hand integrating force-position multimodal perception using a monocular camera

Area of Science:

• Robotics
• Human-Machine Interaction
• Sensor Technology

Background:

• Anthropomorphic hands are vital for human-machine interaction.
• Existing high-DOF hands often lack multimodal perception or are prohibitively expensive.
• There is a need for cost-effective, dexterous hands with advanced sensing capabilities.

Purpose of the Study:

• To develop a dexterous anthropomorphic hand with multimodal sensing capabilities.
• To achieve high degrees of freedom (DOF) in a cost-effective manner.
• To demonstrate multimodal perception using only a camera.

Main Methods:

• Designed a dexterous hand with 18 DOF, eliminating the need for position or force sensors.
• Integrated an actuation and sensing forearm system.
• Developed a multimodal perception model based on camera tracking of tendon displacement and tension, coupled with hand dynamics.

Main Results:

• The hand successfully achieved 18 DOF without traditional sensors.
• Simultaneous actuation and multimodal sensing (joint angles, fingertip position/force, object properties) were realized.
• The camera-based tendon tracking effectively provided rich sensory information.

Conclusions:

• The developed hand offers a cost-effective solution for multimodal sensing in robotics.
• This approach demonstrates the potential of camera-based sensing for dexterous robotic hands.
• The system shows promise for advanced human-machine interaction applications.