Hand-Object Contact Force Estimation from Markerless Visual Tracking.

Area of Science:

• Robotics
• Computer Vision
• Physics

Background:

• Traditional force measurement relies on intrusive, expensive force transducers.
• Estimating interaction forces from vision alone is challenging due to inherent ambiguities in multi-contact scenarios.

Purpose of the Study:

• To develop a cost-effective, non-intrusive method for estimating manipulation contact forces using only visual data.
• To address limitations of inverse optimization models in force estimation.

Main Methods:

• Collected and released the first large-scale dataset of manipulation kinodynamics (3.2 hours).
• Utilized recurrent neural networks (RNNs) to map kinematic features to manipulation forces.
• Refined RNN predictions using physics-based optimization with second-order cone programming (SOCP).

Main Results:

• Demonstrated successful estimation of interaction forces using a single RGB-D camera.
• Achieved force estimations consistent with observed motion and human manipulation intuition.
• The developed method provides a reliable and accurate alternative to traditional force sensing.

Conclusions:

• Vision-based force estimation is feasible and effective for manipulation tasks.
• The novel dataset and RNN-based approach advance the field of robotic sensing.
• This work offers a practical solution for robots to understand and replicate human-like manipulation.