一个基于深度学习的框架,用于音乐同步的舞蹈编舞,以及姿势定量和运动预测,用于活动识别.
Weiwei Fan1,2, Xuerui An3
1College of Art, Southwest Minzu University, Chengdu, 610200, China.
Scientific reports
|October 24, 2025
概括
这项研究引入了一个深度学习框架,用于与音乐同步的AI生成的舞蹈编舞. 该模型增强了运动合成,并在虚拟现实和交互式娱乐中有应用.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 动态舞蹈编舞创作在娱乐和虚拟现实等各个领域都有应用.
- 现有的方法缺乏复杂的音乐同步和表达式的运动合成.
研究的目的:
- 为音乐同步舞蹈编舞开发一个深度学习框架.
- 为了提高运动合成质量和音乐-运动相关性.
主要方法:
- 利用修改的视觉转换器来提取姿势和生成骨架图.
- 采用修改图形卷积网络进行时空联合关系分析.
- 应用K-平均集群和向量定量化变量自动编码器用于姿势数据离散.
- 优化了音乐同步,使用节拍对齐损失和差异演变算法来调节重量.
主要成果:
- 获得了最低的Fréchet发射距离 (FID) 分数 (FIDk=32.451,FIDg=11.219) 和高的音乐运动相关性 (0.341).
- 与最先进的技术相比,演示了增强的运动合成.
- 获得了高分类准确度 (97.019%) 与高效的计算性能 (0.8431G FLOPs).
结论:
- 提出的深度学习框架有效地产生了与音乐同步的舞蹈编舞.
- 该框架显示了运动合成和同步精度的显著改进.
- 潜在的应用包括人工智能生成的编舞,虚拟舞蹈指导和交互式娱乐.
相关概念视频
Muscle Coordination and Action
3.0K
Muscle coordination is a complex and finely tuned process essential for smooth and purposeful movements like flexion, extension, adduction, abduction, and rotation. The human body orchestrates the actions of various muscles working in concert, each with a specific role. Four functional types describe how muscles work together: agonist, antagonist, synergist, and fixator.
Agonists
Agonist muscles, often called prime movers, are the primary muscles responsible for producing a specific movement....
Agonists
Agonist muscles, often called prime movers, are the primary muscles responsible for producing a specific movement....
3.0K
Absolute Motion Analysis- General Plane Motion
531
Visualize a drone, with its propellers spinning rapidly, hovering mid-air. The fascinating movements and operations of this drone can be comprehended by applying the principle of general plane motion.
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
531
One-Degree-of-Freedom System
801
In mechanical engineering, one-degree-of-freedom systems form the basis of a wide range of electrical and mechanical components. Using these models, engineers can predict the behavior of various parts in a larger system, which gives them insight into how different forces interact with each other.
A one-degree-of-freedom system is defined by an independent variable that determines its state and behavior. One example of a one-degree-of-freedom system is a simple harmonic oscillator, such as a...
A one-degree-of-freedom system is defined by an independent variable that determines its state and behavior. One example of a one-degree-of-freedom system is a simple harmonic oscillator, such as a...
801
Relative Motion Analysis using Rotating Axes
876
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
876
Relative Motion Analysis using Rotating Axes-Problem Solving
695
Consider a crane whose telescopic boom rotates with an angular velocity of 0.04 rad/s and angular acceleration of 0.02 rad/s2. Along with the rotation, the boom also extends linearly with a uniform speed of 5 m/s. The extension of the boom is measured at point D, which is measured with respect to the fixed point C on the other end of the boom. For the given instant, the distance between points C and D is 60 meters.
Here, in order to determine the magnitude of velocity and acceleration for point...
Here, in order to determine the magnitude of velocity and acceleration for point...
695
Introduction to Learning
931
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
931


