Cutting Pose Prediction from Point Clouds.

Area of Science:

• Robotics and Automation
• Computer Vision
• Machine Learning

Background:

• Deep Neural Networks (DNNs) show promise for 3D object understanding using point clouds.
• Existing DNN methods primarily focus on classification and segmentation of rigid objects.
• Natural object interaction is crucial for applications in medicine, agriculture, and automation.

Purpose of the Study:

• To develop a DNN-based method for learning tool placement from human demonstrations in virtual reality.
• To apply this method to a challenging slaughterhouse cutting task requiring local geometric understanding.
• To investigate the accuracy and generalization capabilities of the proposed approach.

Main Methods:

• Utilized a PointNet-based architecture for processing point cloud data.
• Employed virtual reality for human demonstrations of tool placement.
• Introduced an intermediate five-Degree of Freedom (DoF) cutting plane representation (point and normal vector).

Main Results:

• Achieved 8 out of 11 acceptable cuts in a live experiment.
• Reduced average translation error from 1.5 cm to 0.8 cm and orientation error from 4.59° to 4.48° with further training.
• Demonstrated promising generalization on a similar slaughterhouse task and the LINEMOD dataset.

Conclusions:

• The proposed method effectively learns tool placement for complex tasks using point cloud data and virtual reality demonstrations.
• Improvements in training data and DNN architecture significantly enhance accuracy.
• The approach shows potential for real-world applications requiring precise manipulation of natural objects.