DiFusion: Flexible Stylized Motion Generation Using Digest-and-Fusion Scheme

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Existing text-driven human motion synthesis methods struggle with style expression.
• Controlling motion style through diverse modalities remains a challenge.

Purpose of the Study:

• To propose DiFusion, a novel framework for diversely stylized human motion generation.
• To enable flexible control over motion content via text and style via multiple modalities (textual labels or motion sequences).

Main Methods:

• Employs a dual-condition motion latent diffusion model for independent content and style control.
• Introduces the Digest-and-Fusion training scheme to handle imbalanced dataset complexities.
• Digests domain-specific knowledge and adaptively fuses it for compatible training.

Main Results:

• Demonstrates superior performance over existing approaches in content alignment, style expressiveness, realism, and diversity.
• Comprehensive evaluations validate the effectiveness of the DiFusion framework.
• The method shows potential for practical applications like motion style interpolation.

Conclusions:

• DiFusion offers a robust solution for stylized human motion synthesis with flexible multi-modal style control.
• The Digest-and-Fusion training scheme effectively addresses dataset imbalances.
• The framework significantly advances the state-of-the-art in human motion generation.