标志语言动作生成来自标志特征的动作
Manuel Gil-Martín1, María Villa-Monedero1, Andrzej Pomirski2
1Grupo de Tecnología del Habla y Aprendizaje Automático (T.H.A.U. Group), Information Processing and Telecommunications Center, Escuela Técnica Superior de Ingenieros de Telecomunicación, Universidad Politécnica de Madrid, 28040 Madrid, Spain.
Sensors (Basel, Switzerland)
|December 9, 2023
概括
本研究介绍了一种基于变压器的深度学习模型,用于从符号音符 (汉堡符号系统) 中生成详细的手语运动. 该系统在运动生成和信号结束检测方面实现了高精度.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 计算语言学 计算语言学
背景情况:
- 信号语言的动作生成是复杂的,需要详细的代表手和身体的运动.
- 现有的方法可能缺乏实现现实的手语合成所需的细粒度.
- 符号音标提供了一种结构化的方法来编码基本的符号特征.
研究的目的:
- 提出和评估一种新的深度学习架构,用于从符号语音生成手语运动.
- 开发和评估一个停止检测模块,以准确地终止信号序列.
- 通过各种策略来优化系统性能,如填充,插值和数据增强.
主要方法:
- 利用基于变压器的深度学习架构来生成信号运动.
- 使用HamNoSys (汉堡符号系统) 进行详细的符号和语音表示.
- 实现动态时间曲线 (DTW) 用于运动生成评估和ROC曲线用于停止检测准确性.
主要成果:
- 最好的系统配置实现了每平均DTW距离为0.1057的运动生成.
- 停止检测模块显示ROC曲线下的面积 (AUC) 大于0.94.
- 评估了多种策略,包括填充,插值和数据增强,以找到最佳配置.
结论:
- 拟议的基于变压器的方法有效地从音符中产生详细的手语运动.
- 集成的停止检测模块准确地预测信号序列的结束.
- 这项研究为自动手语运动合成提供了一个强大的框架.
相关概念视频
Signal Flow Graphs
226
Signal-flow graphs offer a streamlined and intuitive approach to representing control systems, providing an alternative to traditional block diagrams. These graphs use branches to symbolize systems and nodes to represent signals, effectively illustrating the relationships and interactions within the system.
In a signal-flow graph, branches denote the system's transfer functions, while nodes represent the signals. The direction of signal flow is indicated by arrows, with the corresponding...
In a signal-flow graph, branches denote the system's transfer functions, while nodes represent the signals. The direction of signal flow is indicated by arrows, with the corresponding...
226
Relative Motion Analysis - Velocity
368
A stroke engine has a slider-crank mechanism that converts rotational motion from the crank into linear motion of the slider or vice versa. This mechanism consists of three main parts: the crank, the connecting rod, and the slider.
When an external force is exerted, it sets the crank into a rotational movement. This, in turn, instigates the motion of the connecting rod, leading to what is referred to as a general plane motion. This process involves two key points - point A on the connecting rod...
When an external force is exerted, it sets the crank into a rotational movement. This, in turn, instigates the motion of the connecting rod, leading to what is referred to as a general plane motion. This process involves two key points - point A on the connecting rod...
368
Relative Motion Analysis - Acceleration
361
A slider-crank mechanism converts rotational motion from the crank into linear motion of the slider or vice versa. This mechanism consists of three main parts: the crank, the connecting rod, and the slider. The movement of the slider-crank is an example of general plane motion as the fluctuating angle between the crank and the connecting rod. Consider a segment AB where point A is at the end of the slider and point B is on the diametrically opposite end to point A, on a crack. The variance in...
361
Relative Motion Analysis using Rotating Axes
469
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
469
Absolute Motion Analysis- General Plane Motion
224
Visualize a drone, with its propellers spinning rapidly, hovering mid-air. The fascinating movements and operations of this drone can be comprehended by applying the principle of general plane motion.
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
224
Relative Motion Analysis using Rotating Axes-Problem Solving
406
Consider a crane whose telescopic boom rotates with an angular velocity of 0.04 rad/s and angular acceleration of 0.02 rad/s2. Along with the rotation, the boom also extends linearly with a uniform speed of 5 m/s. The extension of the boom is measured at point D, which is measured with respect to the fixed point C on the other end of the boom. For the given instant, the distance between points C and D is 60 meters.
Here, in order to determine the magnitude of velocity and acceleration for point...
Here, in order to determine the magnitude of velocity and acceleration for point...
406


