通过变压器学习和巩固F0序列和持续变化的音调的上下文化轮表示
1Department of Information Engineering and Computer Science, Feng Chia University, Taichung 407, Taiwan.
The Journal of the Acoustical Society of America
|November 18, 2024
概括
本研究介绍了TNet-Full,这是普通话音调分类的变压器模型. 它通过结合基本的频率 (F0) 轮和音节节奏,显著提高了准确性,在阅读和对话语言中表现优于基线模型.
科学领域:
- 计算语言学 计算语言学
- 语音处理 语音处理
- 机器学习 机器学习
背景情况:
- 汉语音调分类的传统方法依赖于声学特征 (例如,音频谱) 或预训练模型 (例如, wav2vec2.0).
- 与连接主义时间分类的深度神经网络是常用的,但可以通过新的架构来改进.
研究的目的:
- 提出和评估TNet-Full,一种基于变压器的新型架构,用于增强普通话音调分类.
- 为了利用估计的基本频率 (F0) 和音节边界信息来改进语音识别.
主要方法:
- 开发了TNet-Full,这是一个具有轮和节奏编码器的变压器架构.
- 实现了一个交互编码器,在轮和节奏信息之间进行交叉注意.
- 使用估计的基本频率 (F0) 值和对齐的音节/单词边界信息.
主要成果:
- 与基线变压器相比,TNet-Full实现了阅读语音的24.4% (95.8%准确率) 和对话语音的6.3% (58.4%准确率) 的绝对性能改善.
- 阅读演讲的相对改善为34.2%,对话演讲为12.1%.
- 该模型有效地利用音调轮和节奏信息,以更好地识别音调.
结论:
- 拟议的TNet-Full架构显著提高了普通话音调分类的准确性.
- 整合F0轮和节奏信息,特别是音节时间组织,对于强大的音调识别至关重要.
- 这些发现表明,稳定和可预测的音节时间有助于音调感知,反映了人类的听觉处理.
相关概念视频
01:16Types Of Transformers
948
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
948
01:26Transformers
1.1K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.1K
01:26The Ideal Transformer
356
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
356
01:21Perceiving Loudness, Pitch, and Location
195
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
195
01:25Transformers with Off-Nominal Turns Ratios
139
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
139
01:21Cognitive Learning
220
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
220