Keyframe Control of Music-Driven 3D Dance Generation

Area of Science:

• Computer Graphics and Animation
• Artificial Intelligence
• Machine Learning

Background:

• Current AI methods for 3D dance generation primarily rely on music, offering limited control over motion.
• Lack of precise control hinders the creation of dance choreography that aligns with specific artistic intentions or poses.

Purpose of the Study:

• To develop a novel AI-driven technique for music-driven 3D dance generation with enhanced control.
• To synthesize visually diverse and plausible dance motions that adhere to both musical cues and user-defined key poses.

Main Methods:

• Introduced keyframe interpolation for music-driven dance generation.
• Employed normalizing flows to learn the probability distribution of dance motions conditioned on music and sparse key poses.
• Incorporated time embedding at each timestep for robust transitions between key poses of varying lengths.

Main Results:

• The proposed model generates more realistic, diverse, and beat-matching dance motions compared to state-of-the-art methods.
• Qualitative and quantitative experiments confirm the superiority of the keyframe-based control approach.
• Demonstrated improved diversity in generated dance motions through keyframe-based control.

Conclusions:

• Keyframe interpolation offers a superior method for controlling AI-generated 3D dance choreography.
• The novel transition generation technique effectively synthesizes plausible and diverse dance motions synchronized with music and key poses.
• This approach significantly enhances the controllability and quality of AI-assisted dance creation for 3D animation.