DanceAgent: Dance Movement Refinement With LLM Agent.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Current text-to-motion synthesis models often overlook fine-grained motion details and quality refinement.
• Existing models like MotionGPT lack multi-turn interaction and multi-modality integration, limiting their effectiveness in complex tasks like dance training.

Purpose of the Study:

• To develop a sophisticated text-to-motion synthesis framework capable of multi-turn interaction for dance training.
• To address the need for assessing motion performance and providing refinement through textual instructions and motion-based feedback.

Main Methods:

• Collected two novel fine-grained dance datasets (jazz and folk dance).
• Implemented global and local optimization during motion encoding with Hidden Markov Model (HMM) temporal modeling for movement analysis.
• Proposed a multi-turn dialogue framework for three-stage generation: motion assessment, text instructions, and motion refinement.

Main Results:

• The proposed method significantly outperforms existing approaches in both quantitative and qualitative metrics on jazz and folk dance datasets.
• Established a new benchmark for motion-text generation specifically tailored for dance training applications.
• Demonstrated the framework's effectiveness in assisting dance beginners with movement feedback and refinement.

Conclusions:

• The developed multi-turn framework enhances text-to-motion synthesis by incorporating fine-grained motion details and interactive refinement.
• This approach offers a promising solution for improving dance training through AI-powered feedback and personalized guidance.
• The study highlights the potential of advanced AI models in specialized domains requiring nuanced motion understanding and generation.