Multimodality Driven Impedance-Based Sim2Real Transfer Learning for Robotic Multiple Peg-in-Hole Assembly

Area of Science:

• Robotics
• Artificial Intelligence
• Smart Manufacturing

Background:

• Robotic rigid contact-rich manipulation is crucial for smart manufacturing.
• Reinforcement learning (RL) has improved single peg-in-hole assembly but struggles with multiple peg-in-hole tasks due to complex constraints.
• Existing solutions for multiple peg-in-hole assembly lack flexibility for real-world industrial deployment.

Purpose of the Study:

• To design a novel and challenging multiple peg-in-hole assembly setup using the Industrial Metaverse.
• To develop a robust solution scheme for complex robotic assembly tasks.
• To enable flexible and efficient transfer of learned policies to real industrial scenarios.

Main Methods:

• Utilized the Industrial Metaverse for a novel multiple peg-in-hole assembly setup.
• Integrated multi-modal sensory inputs (vision, proprioception, force/torque) for compact representation learning.
• Employed reinforcement learning in simulation, with domain randomization and impedance control for sim-to-real transfer.

Main Results:

• Successfully demonstrated effective multiple peg-in-hole assembly in real-world scenarios.
• Achieved policy transfer from simulation to reality without additional real-world exploration.
• Showcased generalization capabilities across different object shapes.

Conclusions:

• The proposed Industrial Metaverse-based approach effectively addresses challenges in robotic multiple peg-in-hole assembly.
• Multi-modal learning and sim-to-real transfer techniques enhance sample efficiency and real-world applicability.
• The solution offers a flexible and robust method for smart manufacturing applications.