MsDUNE:通过迪里克莱特不确定性估计,为独立于扬声器的口唇阅读提供一个多尺度的掩盖时间融合框架
Jinghan Wu1, Xingwei An2, Yakun Zhang3
1Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, 300072, China; Tianjin Artificial Intelligence Innovation Center, Tianjin, 300450, China.
概括
本研究介绍了MsDUNE,这是一个新的唇读框架,可以提高看不见的演讲者的准确性. 它有效地减少了个体之间的特征分布的差异,增强了概括能力.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 由于特征分布的变化,看嘴唇的表现在看不见的扬声器上显著下降.
- 传统的唇读模型需要大量的标记数据,限制了它们的适应性.
研究的目的:
- 为了引入一个新的框架,多尺度的掩盖时间融合与迪里克莱特不确定性估计 (MsDUNE),以解决口唇阅读的扬声器变化.
- 增强唇读模型的概括能力,用于看不见的和重叠的扬声器.
主要方法:
- MsDUNE使用迪里克莱分布来参数化特征分支的潜空间,通过证据和信念质量进行评估.
- 该框架通过考虑两个分支之间的特征信念的相互影响来校准多级特征分布.
主要成果:
- MsDUNE在LRW-ID,AV Letters和CVSR100数据集上展示了最先进的性能.
- 该方法在未见或重叠的扬声器的情况下显著提高了读唇精度.
结论:
- MsDUNE有效地减轻了扬声器之间的特征分布差异,从而改善了唇读普遍化.
- 拟议的框架提供了一个强大的解决方案,用于在现实世界中提高唇读性能,各种扬声器场景.
相关概念视频
01:19Masking and Demasking Agents
2.7K
EDTA titrations may necessitate masking and demasking agents to temporarily protect a particular metal ion in a mixture from the EDTA reaction. These agents facilitate the sequential analysis of the metal ions by forming stable complexes with some—but not all—metal ions during certain steps.
There are many masking agents, such as cyanide, fluoride, triethanolamine, thiourea, and 2,3-bis(sulfanyl)propan-1-ol (formerly 2,3-dimercapto-1-propanol), with the masking agent chosen based on...
There are many masking agents, such as cyanide, fluoride, triethanolamine, thiourea, and 2,3-bis(sulfanyl)propan-1-ol (formerly 2,3-dimercapto-1-propanol), with the masking agent chosen based on...
2.7K
00:59Uncertainty: Overview
990
In analytical chemistry, we often perform repetitive measurements to detect and minimize inaccuracies caused by both determinate and indeterminate errors. Despite the cares we take, the presence of random errors means that repeated measurements almost never have exactly the same magnitude. The collective difference between these measurements - observed values - and the estimated or expected value is called uncertainty. Uncertainty is conventionally written after the estimated or expected value.
990
01:24Facial Feedback Hypothesis
253
Charles Darwin proposed that facial expressions are an evolutionary adaptation for communication. He argued that these expressions are not influenced by culture but are universal across species. For example, a snarling expression with exposed teeth signals a threat in many animals, including humans. Darwin also suggested that displaying an emotion can intensify the feeling. Smiling, for example, could enhance one's sense of happiness. This idea laid the foundation for understanding the role...
253
01:21Perceiving Loudness, Pitch, and Location
436
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
436