星:骨意识的基于文本的4D化身生成与网络内运动重定位
IEEE transactions on visualization and computer graphics
|April 11, 2025
概括
我们介绍了STAR,这是一种用于从文本中生成动画4D化身的新方法. 通过使用骨架意识的运动重定位和混合Score蒸采样模块来克服当前方法的局限性,以提高动画质量.
科学领域:
- 计算机视觉 计算机视觉
- 计算机图形 计算机图形
- 人工智能的人工智能
背景情况:
- 目前用于从文本生成4D化身的方法依赖于文本到图像模型,这些模型面临域间隙和视图一致性方面的挑战.
- 后期动画技术通常会导致翻译文物和错位,原因是avatar几何和运动的差异.
研究的目的:
- 开发一个强大的框架,直接从文本描述中生成高质量的4D动画化身.
- 解决现有的优化通过动画模式在合成和动画3D化身的局限性.
主要方法:
- 拟议的骨架意识的基于文本的4D化身生成与网络内运动重定位 (STAR).
- 整合了骨架意识的运动重定位,以纠正基于几何和骨架差异的不匹配的源运动.
- 使用混合分数蒸采样 (SDS) 模块,具有骨架条件的文本到图像 (T2I) 和文本到视频 (T2V) 前置,用于多视图和框架一致的监控.
主要成果:
- STAR允许对avatar的几何,纹理和运动进行端到端的优化.
- 演示了高质量的4D化身与与文本描述一致的生动动画的合成.
- 定量和定性实验证实了拟议方法的有效性.
结论:
- 星星有效地生成高保真度的4D化身,具有准确和自然的动画.
- 拟议的方法克服了以前的文字到4D化身生成技术的关键局限性.
- 废弃性研究验证了STAR框架内单个成分的贡献.
更多相关视频
06:20Author Spotlight: Development of an Automated Camera-Based System for Real-Time Blast Overpressure Monitoring and TBI Risk Assessment in Military Training
Published on: December 6, 2024
2.3K
10:14Author Spotlight: Enhancing Neurorehabilitation Through EEG, Motor Imagery, and Virtual Reality
Published on: May 10, 2024
805
相关概念视频
Carbon Skeletons
84.7K
Life on Earth is carbon-based, as all macromolecules that make up living organisms contain carbon atoms. All organic compounds have a carbon backbone. Each carbon atom is tetravalent and can bond with four other atoms, making it an extraordinarily flexible component of biological molecules. Because carbon’s valence electrons are stable, it rarely becomes an ion. As the carbon chain increases in length, structural modifications such as ring structures, double bonds, and branching side...
84.7K
Bone Structure
40.4K
Within the skeletal system, the structure of a bone, or osseous tissue, can be exemplified in a long bone, like the femur, where there are two types of osseous tissue: cortical and cancellous.
40.4K
Bone Remodeling
34.4K
Bone remodeling is a continuous and balanced process of bone resorption by osteoclasts and bone formation by osteoblasts. In adults, it helps maintain bone mass and calcium homeostasis. While mechanical stress can stimulate turnover as part of the normal maintenance and reparative process, several hormones also regulate bone remodeling.
34.4K
Bone Formation by Intramembranous Ossification
16.2K
Intramembranous ossification is one of the two processes involved in the development of bones within an embryo. The flat bones of the face, most of the cranial bones, and the clavicles are formed via this process. During intramembranous ossification, the bones develop directly from sheets of undifferentiated mesenchymal connective tissue.
The process begins when mesenchymal cells in the embryonic skeleton gather together and differentiate into osteogenic cells, which then develop into ...
The process begins when mesenchymal cells in the embryonic skeleton gather together and differentiate into osteogenic cells, which then develop into ...
16.2K
Three-Dimensional Force System
3.2K
In mechanical engineering, a three-dimensional force system is a system of forces acting in three dimensions, with forces applied along the x, y, and z coordinate axes. The three-dimensional force system is an important concept in mechanical engineering, as it allows engineers to understand and analyze the behavior of objects and structures in three dimensions. By understanding the forces acting on a system, engineers can design more efficient and effective mechanical systems that can withstand...
3.2K
Bone Formation by Endochondral Ossification
14.7K
Bone formation, or ossification, begins around the sixth to seventh week of embryonic development. Most bones develop from a cartilaginous template through the process of endochondral ossification. Cartilage formation begins when clusters of mesenchymal cells differentiate into chondrocytes. These chondrocytes proliferate rapidly and secrete an extracellular matrix that becomes encased in a membrane called the perichondrium. The resulting cartilage model provides a template that resembles the...
14.7K
