从三轴加速度计信号的语音合成,使用基于调整器的深度神经网络
Jinuk Kwon1, Jihun Hwang1, Jee Eun Sung2
1Department of Electronic Engineering, Hanyang University, Seoul, South Korea.
Computers in biology and medicine
|September 4, 2024
概括
这项研究探讨了使用加速度计从面部运动合成语音. 添加第五个传感器显著提高了语音合成质量,为实际的无声语音接口铺平了道路.
科学领域:
- 生物医学工程 生物医学工程
- 语音技术 语言技术
- 信号处理 信号处理
背景情况:
- 无声语音接口 (SSI) 提供非声学通信,之前的研究表明基于加速度计的系统具有前景.
- 基于加速度计的SSI在识别默默说的话方面超过了表面肌电学 (sEMG) 系统.
研究的目的:
- 从三轴加速度计信号中合成口语的可行性.
- 评估基于加速度计的SSI在实际静音通信应用中的潜力.
主要方法:
- 19名健康参与者在脸上戴着5个加速度计,以捕捉与语音相关的动作.
- 一个卷积增强的变压器 (Conformer) 模型将加速度计信号转换为Mel谱图,通过HiFi-GAN合成音频.
- 使用十倍交叉验证和Mel cepstral扭曲 (MCD) 来评估Mel光谱质量.
主要成果:
- 通过四个优化的加速度计,平均MCD达到5.03±0.65.
- 在下巴下添加第五个加速度计显著改善了Mel光谱质量,平均MCD为4.86 ± 0.65 (p < 0.001).
- 结果超过了使用较少传感器和可比的训练数据的传统SSI.
结论:
- 从加速度计信号合成语音是可行的,并显示高质量的.
- 拟议的方法为现有的SSI提供了一种低功耗,便携性和抗工件的替代方案.
- 这一进步可能会推动基于加速度计的无声语音接口的更广泛采用.
相关概念视频
Relative Motion Analysis using Rotating Axes - Acceleration
327
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame. The absolute velocity of point B is determined by adding the absolute velocity of point A, the relative velocity of point B in the rotating frame, and the effects caused by the angular velocity within the rotating frame.
Time differentiation is...
Time differentiation is...
327
Gyroscope
2.9K
A gyroscope is defined as a spinning disk in which the axis of rotation is free to assume any orientation. When spinning, the orientation of the spin axis is unaffected by the orientation of the body that encloses it. The body or vehicle enclosing the gyroscope can be moved from place to place, while the orientation of the spin axis remains the same. This makes gyroscopes very useful in navigation, especially where magnetic compasses cannot be used, such as in crewed and crewless spacecraft,...
2.9K
Relative Motion Analysis - Acceleration
343
A slider-crank mechanism converts rotational motion from the crank into linear motion of the slider or vice versa. This mechanism consists of three main parts: the crank, the connecting rod, and the slider. The movement of the slider-crank is an example of general plane motion as the fluctuating angle between the crank and the connecting rod. Consider a segment AB where point A is at the end of the slider and point B is on the diametrically opposite end to point A, on a crack. The variance in...
343
Gyroscope: Precession
4.0K
Precession can be demonstrated effectively through a spinning top. If a spinning top is placed on a flat surface near the surface of the Earth at a vertical angle and is not spinning, it will fall over due to the force of gravity producing a torque acting on its center of mass. However, if the top is spinning on its axis, it precesses about the vertical direction, rather than topple over due to this torque. Precessional motion is a combination of a steady circular motion of the axis and the...
4.0K
Relative Motion Analysis using Rotating Axes
450
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
450
Three-Dimensional Force System
2.0K
In mechanical engineering, a three-dimensional force system is a system of forces acting in three dimensions, with forces applied along the x, y, and z coordinate axes. The three-dimensional force system is an important concept in mechanical engineering, as it allows engineers to understand and analyze the behavior of objects and structures in three dimensions. By understanding the forces acting on a system, engineers can design more efficient and effective mechanical systems that can withstand...
2.0K


