Learning Haptic-Based Object Pose Estimation for In-Hand Manipulation Control With Underactuated Robotic Hands

Area of Science:

• Robotics
• Haptic Sensing
• Underactuated Systems

Background:

• Underactuated compliant robotic hands present modeling challenges, often relying on vision for pose estimation.
• Visual perception is limited in occluded or partially occluded environments, hindering robotic manipulation.
• Existing methods struggle with uncertainty and environmental limitations.

Purpose of the Study:

• To explore the use of haptics (kinesthetic and tactile sensing) for pose estimation and in-hand manipulation with underactuated robotic hands.
• To develop a vision-free feature state representation using low-cost hardware.
• To enable robust object manipulation in environments where visual line-of-sight is compromised.

Main Methods:

• Development of a low-cost, 3D-printable flexible tactile sensor for implicit contact information.
• Analysis of kinesthetic and tactile features with various regression models for pose prediction using a two-finger underactuated hand.
• Implementation of a Model Predictive Control (MPC) approach for haptics-only object manipulation.

Main Results:

• Successful pose estimation for objects with diverse geometry, stiffness, and texture.
• Demonstration of accurate object manipulation to desired workspace goals using solely haptic feedback.
• Validation of the proposed haptic approach in mitigating occluded environment challenges.

Conclusions:

• Haptic sensing offers a viable alternative to vision for pose estimation and manipulation with underactuated robotic hands.
• The proposed low-cost tactile sensor and MPC framework enable robust, vision-free in-hand manipulation.
• This research advances robotic capabilities in complex and occluded environments.