MAGIC: Meta-Ability Guided Interactive Chain-of-Distillation for Effective-and-Efficient Vision-and-Language...

Area of Science:

• Artificial Intelligence
• Robotics
• Computer Vision

Background:

• Large models in Embodied AI (E-AI) face challenges with parameter size and computational demands, hindering robotics integration.
• Vision-and-Language Navigation (VLN) is a key E-AI task requiring efficient and capable agents.

Purpose of the Study:

• To develop lightweight student models for E-AI tasks, specifically VLN, using knowledge distillation.
• To propose a novel Meta-Ability Guided Interactive Chain-of-distillation (MAGIC) method for efficient model training.

Main Methods:

• Proposed the Meta-Ability Knowledge Distillation (MAKD) framework to refine agent meta-abilities.
• Introduced Meta-Knowledge Randomization Weighting (MKRW) and Meta-Knowledge Transferable Determination (MKTD) for dynamic weight adjustment.
• Implemented an Interactive Chain-of-Distillation (ICoD) strategy for multi-step teacher-student co-evolution.

Main Results:

• The smallest model, MAGIC-S (5% of teacher size), outperformed prior methods on the R2R test unseen leaderboard.
• The largest model, MAGIC-L, improved state-of-the-art performance by 5.84% in SPL and 3.18% in SR.
• MAGIC-S demonstrated superior performance and real-time efficiency on a newly collected dataset from living environments.

Conclusions:

• Knowledge distillation is highly effective for creating efficient VLN agents.
• The MAGIC method offers a promising direction for developing practical and high-performing E-AI systems.
• The proposed approach enables significant model compression without sacrificing performance, facilitating real-world robotic applications.