Trajectory Tracking of Variable Centroid Objects Based on Fusion of Vision and Force Perception

Area of Science:

• Robotics
• Computer Vision
• Control Systems

Background:

• Dynamic throwing and catching of rigid objects by robots is well-established.
• Predicting and tracking the in-flight trajectories of nonrigid objects with variable centroids presents significant challenges due to their unpredictable motion.

Purpose of the Study:

• To develop a robust robot control system for precise prediction and tracking of nonrigid object trajectories.
• To enhance robot perception by fusing vision and force information for improved dynamic manipulation tasks.

Main Methods:

• A Variable Centroid Trajectory Tracking Network (VCTTN) was proposed, integrating force data into a vision neural network.
• A model-free robot control system based on VCTTN was developed.
• A dataset of flight trajectories for variable centroid objects was generated using a robot arm for training and validation.

Main Results:

• The vision-force VCTTN demonstrated superior performance in trajectory prediction and tracking compared to traditional vision-only perception methods.
• The VCTTN-based system achieved highly precise prediction and tracking using partial in-flight vision data.
• Excellent tracking performance was observed in experimental evaluations.

Conclusions:

• Fusing vision and force information significantly improves the robot's ability to handle dynamic manipulation of nonrigid objects.
• The proposed VCTTN offers a promising approach for advanced robotic grasping and manipulation of objects with complex dynamics.