一个基于自我监督的语音表示的语音感知相似空间
Bronya R Chernyak1, Ann R Bradlow2, Joseph Keshet1
1Faculty of Electrical & Computer Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel.
The Journal of the Acoustical Society of America
|June 21, 2024
概括
这项研究引入了一种分析语音的新方法,使用感知相似空间而不是传统的声学特征. 这种方法更好地解释了语音可理解性的差异,特别是对于第二语言 (L2) 发言者.
科学领域:
- 语音处理 语音处理
- 声学语音学的声音学.
- 机器学习 机器学习
背景情况:
- 语音识别系统在不理想的条件下扎,例如背景噪音和第二语言 (L2) 语音.
- 现有的分析语音的光谱-时间性质的方法使得许多可理解性变化无法解释.
- 需要新的方法来理解语音变化,并提高识别准确性.
研究的目的:
- 研究一种使用感知相似空间进行语音分析的替代方法.
- 确定感知相似性是否可以解释L1和L2语言的可理解性变化.
- 评估自我监督学习对编码语音区别的潜力.
主要方法:
- 利用自主监督学习来创建语音样本的感知相似空间.
- 编码语音区别,而不依赖于预定义的声学特征或语音与文本对齐.
- 在这个空间中,第一语言 (L1) 和L2英语语音样本之间的量化距离.
主要成果:
- 与L1样本相比,L2英语语音样本在感知空间中显示出更大的变化 (较少紧密的聚类).
- 感知相似空间内的距离与人类听众的表现相关联.
- 对于L1英语听众来说,他们的识别准确性较低,而对于L2讲者来说,他们的讲话在空间上更遥远.
结论:
- 通过自我监督学习捕获的感知相似性,为语音分析提供了一个强大的新维度.
- 这种方法提供了比传统的基于特征的方法更全面的解释可理解性变化.
- 感知相似空间可能成为下一代语音和语言分析工具的基础.
相关概念视频
01:01Perception of Sound Waves
4.4K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
4.4K
01:21Perceiving Loudness, Pitch, and Location
205
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
205
02:45Self-Discrepancy Theory
18.3K
One influential perspective on what motivates people's behavior is detailed in Tory Higgin's self-discrepancy theory (Higgins, 1987). He proposed that people hold disagreeing internal representations of themselves that lead to different emotional states.
18.3K
01:12Perceptual Constancy
382
Perceptual constancy is the ability to recognize that objects remain consistent and unchanged even when their appearance varies due to changes in sensory input. There are four main types of perceptual constancy: size constancy, shape constancy, color constancy, and brightness constancy.
Size constancy is the recognition that an object remains the same size, even when its image on the retina changes. For instance, a bus is perceived to be large enough to carry people, even if it looks tiny from...
Size constancy is the recognition that an object remains the same size, even when its image on the retina changes. For instance, a bus is perceived to be large enough to carry people, even if it looks tiny from...
382
02:13The Representativeness Heuristic
15.8K
The representative heuristic describes a biased way of thinking, in which you unintentionally stereotype someone or something. For example, you may assume that your professors spend their free time reading books and engaging in intellectual conversation, because the idea of them spending their time playing volleyball or visiting an amusement park does not fit in with your stereotypes of professors.
15.8K
02:16Stereotype Content Model
14.7K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.7K