Acquisition and Neural Network Prediction of 3D Deformable Object Shape Using a Kinect and a Force-Torque Sensor

Area of Science:

• Robotics
• Computer Vision
• Machine Learning

Background:

• Realistic deformation modeling is challenging for objects lacking simple elastic properties.
• Existing methods struggle to capture complex, non-linear deformations.
• Data-driven approaches offer a promising avenue for deformation prediction.

Purpose of the Study:

• To propose a data-driven neural network approach for capturing and predicting object deformations.
• To develop a method that represents deformations without prior knowledge of material properties.
• To enable accurate prediction of object shape changes under external forces.

Main Methods:

• Utilizing 3D point clouds from a Kinect sensor and force-torque sensor data.
• Employing neural gas fitting for deformation representation on a simplified 3D surface.
• Training feedforward neural networks to map force parameters to shape changes.

Main Results:

• Achieved high perceptual similarity in deformed areas (96.6%) and overall shape preservation (86%).
• Reduced mesh complexity by using an average of 40% of the original vertices.
• Demonstrated accurate prediction of object deformations under unknown interactions.

Conclusions:

• The proposed neural gas fitting and neural network approach effectively models and predicts deformations of complex objects.
• This method offers a robust, data-driven solution for real-time deformation analysis in robotics and simulation.
• The approach generalizes well to unseen interactions, advancing the field of deformable object manipulation.