ToggleMimic: A Two-Stage Policy for Text-Driven Humanoid Whole-Body Control

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Natural language is crucial for seamless human-robot interaction and integration into daily life.
• Current imitation learning methods for robots struggle with high-level semantic instructions and dynamic action switching.

Purpose of the Study:

• To develop an end-to-end imitation learning framework for generating robotic motions from textual instructions.
• To enable language-driven multi-task control in humanoid robots.

Main Methods:

• Proposed ToggleMimic, an imitation learning framework combining two-stage policy distillation, cross-attention mechanism, and a gating network.
• Policy distillation bridges the sim-to-real gap.
• Cross-attention enables interpretable text-to-action mapping.
• Gating network improves robustness to linguistic variations.

Main Results:

• ToggleMimic demonstrates effectiveness, generalization capability, and robust text-guided control.
• Framework successfully generates robotic motions from textual instructions.
• Achieved efficient, interpretable, and scalable learning for cross-modal semantic-driven robot control.

Conclusions:

• ToggleMimic offers an efficient, interpretable, and scalable learning paradigm for autonomous robot control.
• The framework enables robots to understand and execute natural language commands for complex tasks.
• This research advances natural language understanding and control in humanoid robots.