MLUG: Bootstrapping Language-Motion Pre-Training for Unified Motion-Language Understanding and Generation

Area of Science:

• Computer Vision
• Animation
• Artificial Intelligence

Background:

• Generating human motion from text is challenging.
• Existing models struggle with open-vocabulary and emotional expressiveness.

Purpose of the Study:

• Introduce MLUG, a new framework for human motion synthesis.
• Leverage vision-language pre-training for improved motion generation.

Main Methods:

• Utilize a unimodal encoder with motion-text contrastive loss.
• Employ a motion-grounded text encoder and decoder.
• Incorporate a motion length predictor for accurate synthesis.

Main Results:

• MLUG effectively aligns textual descriptions with dynamic motion sequences.
• Demonstrates superior performance in generating realistic and diverse motions.
• Sets a new benchmark for open-vocabulary motion generation.

Conclusions:

• MLUG offers an innovative solution for nuanced human motion synthesis.
• The framework enhances emotional expressiveness and physical plausibility.
• MLUG represents a significant advancement in computer vision and animation.