从动作捕捉数据中解码prosodic信息:共同语音手势的严重性
Jacob P Momsen1,2, Seana Coulson1,3
1Joint Doctoral Program in Language and Communication Disorders, San Diego State University and UC San Diego.
Open mind : discoveries in cognitive science
|May 8, 2025
概括
了解身体运动如何与语言同步是关键. 这项研究表明,分析运动是有效的.
科学领域:
- 多式联运通信 多式联运通信
- 语音的生物力学
- 人与计算机的交互
背景情况:
- 来自身体运动的视觉线索,特别是共同语音手势,影响语音感知.
- 对于语音整合相关的手势的特定动态特征在很大程度上是未知的.
- 量化运动动力学和声声学之间的关系至关重要.
研究的目的:
- 调查共同语音手势的动态特征与语音整合具有相关性.
- 用机器学习来量化共同语音手势,以建模语音声学.
- 确定人类运动的动态描述是否可以模拟它们与语言的时间关系.
主要方法:
- 采用机器学习技术来量化共同语音手势.
- 进行了两项实验,操纵了运动动力学和引力加速度之间的关系.
- 分析了同语运动与向下引力力量的异构关系.
主要成果:
- 量化与引力相对的共同语音运动显著改善了语音表达的预测.
- 使用运动动力学有效地预测了语音声学的低通封面.
- 突出或掩盖运动和重力之间的联系影响了预测准确性.
结论:
- 运动动力学,特别是它们与重力的异构关系,与语音手势同步相关.
- 生物力学为理解视听语音信号的整合提供了一个框架.
- 这些发现激励了对视听整合和生物运动感知的进一步研究.
相关概念视频
Absolute Motion Analysis- General Plane Motion
193
Visualize a drone, with its propellers spinning rapidly, hovering mid-air. The fascinating movements and operations of this drone can be comprehended by applying the principle of general plane motion.
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
193
Relative Motion Analysis - Velocity
330
A stroke engine has a slider-crank mechanism that converts rotational motion from the crank into linear motion of the slider or vice versa. This mechanism consists of three main parts: the crank, the connecting rod, and the slider.
When an external force is exerted, it sets the crank into a rotational movement. This, in turn, instigates the motion of the connecting rod, leading to what is referred to as a general plane motion. This process involves two key points - point A on the connecting rod...
When an external force is exerted, it sets the crank into a rotational movement. This, in turn, instigates the motion of the connecting rod, leading to what is referred to as a general plane motion. This process involves two key points - point A on the connecting rod...
330
Relative Motion Analysis using Rotating Axes
433
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
433
Relative Motion Analysis using Rotating Axes-Problem Solving
374
Consider a crane whose telescopic boom rotates with an angular velocity of 0.04 rad/s and angular acceleration of 0.02 rad/s2. Along with the rotation, the boom also extends linearly with a uniform speed of 5 m/s. The extension of the boom is measured at point D, which is measured with respect to the fixed point C on the other end of the boom. For the given instant, the distance between points C and D is 60 meters.
Here, in order to determine the magnitude of velocity and acceleration for point...
Here, in order to determine the magnitude of velocity and acceleration for point...
374
Relative Motion Analysis - Acceleration
313
A slider-crank mechanism converts rotational motion from the crank into linear motion of the slider or vice versa. This mechanism consists of three main parts: the crank, the connecting rod, and the slider. The movement of the slider-crank is an example of general plane motion as the fluctuating angle between the crank and the connecting rod. Consider a segment AB where point A is at the end of the slider and point B is on the diametrically opposite end to point A, on a crack. The variance in...
313
Relative Motion Analysis using Rotating Axes - Acceleration
310
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame. The absolute velocity of point B is determined by adding the absolute velocity of point A, the relative velocity of point B in the rotating frame, and the effects caused by the angular velocity within the rotating frame.
Time differentiation is...
Time differentiation is...
310


