Rhythm is a Dancer: Music-Driven Motion Synthesis With Global Structure.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Computational Creativity

Background:

• Synthesizing human motion, especially with global structure like choreography, is complex.
• Current methods often prioritize local pose smoothness over global context or thematic coherence.
• Integrating music's thematic content into motion synthesis remains a challenge.

Purpose of the Study:

• To develop a music-driven motion synthesis framework for generating long-term, synchronized human dance sequences.
• To ensure generated motions respect specific dance genres and global structural elements.
• To enable motion diversity controlled by music content beyond just beat synchronization.

Main Methods:

• A hierarchical framework with three levels: pose (LSTM for temporal coherence), motif (motion perceptual loss for genre-specific movements), and choreography (genre-based structure selection).
• Utilizing a novel motion perceptual loss to guide movement generation within specific distributions.
• Employing a Long Short-Term Memory (LSTM) component for temporally coherent pose generation.

Main Results:

• The framework successfully generates natural and consistent human motion sequences across various dance types.
• Demonstrated control over the content of synthesized motions, influenced by musical elements.
• Successfully respected the overall global structure and genre of different dance styles.

Conclusions:

• The music-driven framework effectively synthesizes human motion with global structure and genre adherence.
• The system offers control over motion content and diversity, synchronized with music.
• This approach advances music-driven dance synthesis by integrating local, motif, and choreographic levels.