Visual imitation learning from one-shot demonstration for multi-step robot pick and place tasks

Area of Science:

• Robotics
• Machine Learning
• Computer Vision

Background:

• Imitation learning offers intuitive robot programming via human demonstrations.
• Current visual imitation learning requires large datasets, limiting specialized manufacturing applications.

Purpose of the Study:

• To present a one-shot visual imitation learning framework for robots to master multi-step pick & place tasks from a single video.
• To reduce the extensive data requirements typical of current imitation learning methods.

Main Methods:

• The framework integrates hand detection, object detection, and trajectory segmentation.
• Dynamic Movement Primitives (DMPs) are used for skill learning.
• Robot end-effector trajectories are derived from human hand movements in the demonstration.

Main Results:

• The system successfully generalizes to new object positions.
• A significant reduction in data requirements was achieved.
• Reliable reproduction of multi-step pick & place tasks was demonstrated in simulation.

Conclusions:

• One-shot visual imitation learning offers a viable solution for programming industrial robots.
• The proposed framework has the potential to decrease programming complexity and enhance flexibility in manufacturing.