贝兰多++: 3D 舞蹈 GPT 具有编舞记忆
IEEE transactions on pattern analysis and machine intelligence
|September 26, 2023
概括
贝兰多++通过将编舞记忆与生成预训练变压器 (GPT) 结合起来,产生与音乐同步的现实 3D 角色舞蹈. 这种框架确保了时间的连贯性,并遵守了各种音乐类型的舞蹈规范.
科学领域:
- 计算机图形 计算机图形
- 人工智能的人工智能
- 计算创造力的创造力
背景情况:
- 与音乐同步生成现实和时间连贯的3D角色动画是一个复杂的挑战.
- 现有的方法在坚持编舞规范和适应各种音乐类型方面扎.
- 合成类似人类的舞蹈动作需要理解音乐结构和人类运动模式.
研究的目的:
- 提出Bailando++,一个新的音乐到舞蹈框架,用于生成3D角色舞蹈.
- 为了应对编舞规范的挑战,坚持编舞规范,时间连贯性和与各种音乐类型的同步.
- 为了使无人监督的发现可解释的舞蹈风格.
主要方法:
- 开发了一个由两个组件组成的框架:一个编舞记忆来总结舞蹈单元和一个演员批判的生成预训练变压器 (GPT) 用于舞蹈创作.
- 引入了一种演员-批评强化学习方案,用于音乐节奏同步的新节奏对齐奖励功能.
- 嵌入式学习人类舞蹈姿势在旋转领域和音乐上下文编码,以改进运动生成.
主要成果:
- 在质量和数量方面,Bailando++在标准基准上取得了最先进的表现.
- 该框架成功地产生了各种类型的流和音乐一致的舞蹈.
- 在编舞记忆中展示了人类可解释的舞蹈风格姿势的无监督发现.
结论:
- 贝兰多++有效地从音乐中生成同步和风格上适当的3D角色舞蹈.
- 提出的方法,包括强化学习和旋转域姿势学习,显著提高了舞蹈生成质量.
- 该框架为现实的角色动画和创意内容生成提供了一个有前途的方法.
相关概念视频
Planar Rigid-Body Motion
465
Understanding the movement of a rigid body in planar motion involves recognizing that every particle within this body is traversing a path that maintains a consistent distance from a specific plane. This concept is fundamental in the study of physics and mechanical engineering, and it allows us to comprehend better how objects move in space.
Planar motion is typically divided into three distinct categories. The first is rectilinear translation, demonstrated by a subway train that moves along...
Planar motion is typically divided into three distinct categories. The first is rectilinear translation, demonstrated by a subway train that moves along...
465
System of Memory
6.1K
Memory is categorized into three major systems: sensory memory, short-term memory (STM), and long-term memory (LTM). These systems differ in their capacity and the duration for which they can hold information. Sensory memory captures raw sensory input from the environment, holding it for just a few seconds or less. For example, on hearing a brief, loud sound, like a car horn honking, the sound seems to linger in the mind for a moment even after it stops. This is an instance of sensory memory...
6.1K
Muscle Coordination and Action
1.6K
Muscle coordination is a complex and finely tuned process essential for smooth and purposeful movements like flexion, extension, adduction, abduction, and rotation. The human body orchestrates the actions of various muscles working in concert, each with a specific role. Four functional types describe how muscles work together: agonist, antagonist, synergist, and fixator.
Agonists
Agonist muscles, often called prime movers, are the primary muscles responsible for producing a specific movement....
Agonists
Agonist muscles, often called prime movers, are the primary muscles responsible for producing a specific movement....
1.6K
Three-Dimensional Force System:Problem Solving
686
A three-dimensional force system refers to a scenario in which three forces act simultaneously in three different directions. This type of problem is commonly encountered in physics and engineering, where it is necessary to calculate the resultant force on the system, which can then be used to predict or analyze the behavior of the object or structure under consideration.
To solve a three-dimensional force system, first resolve each force into its respective scalar components. Do this using...
To solve a three-dimensional force system, first resolve each force into its respective scalar components. Do this using...
686
Implicit Memories
146
Implicit memories, also known as non-declarative memories, are long-term memories that function outside of conscious awareness. These memories influence behavior and skills without explicit knowledge. This type of memory is evident in tasks like playing tennis, snowboarding, and texting. Implicit memory has three subsystems: procedural memory, conditioning, and priming. This type of memory is essential in various activities, from everyday tasks to specialized skills.
One key aspect of implicit...
One key aspect of implicit...
146
Anatomical Movements
7.2K
Anatomical movements refer to the various actions or motions that can be performed by the body's joints and muscles. These movements are described using specific terms to provide a standardized way of discussing and understanding the range of motion at different joints.
Here are some common anatomical movements:
Flexion and extension motions are in the sagittal (anterior–posterior) plane of motion. These movements take place at the shoulder, hip, elbow, knee, wrist,...
Here are some common anatomical movements:
Flexion and extension motions are in the sagittal (anterior–posterior) plane of motion. These movements take place at the shoulder, hip, elbow, knee, wrist,...
7.2K


