以注意力为基础的超短语音发言和深度视频的融合,用于多模式人体识别
Abderrazzaq Moufidi1,2, David Rousseau2, Pejman Rasti1,2
1Centre d'Études et de Recherche pour l'Aide à la Décision (CERADE), ESAIP, 18 Rue du 8 Mai 1945, 49124 Saint-Barthélemy-d'Anjou, France.
Sensors (Basel, Switzerland)
|July 14, 2023
概括
这项研究引入了一种使用超短语音和唇深视频的新人识别方法,克服了传统生物识别的局限性. 这种新的方法实现了卓越的性能,在具有挑战性的场景中提高了安全性.
科学领域:
- 多模式深度学习 (deep learning) 是一种多模式深度学习.
- 生物识别信息 生物识别信息
- 计算机视觉 计算机视觉
- 语音处理 语音处理
背景情况:
- 传统的生物识别系统通常依赖于长篇演讲或RGB图像,限制了它们的实用性.
- 当扩展的视听数据不可用时,人身识别存在挑战.
研究的目的:
- 开发一种新的多式联络深度学习方法,用于使用超短语音和唇深度视频识别人.
- 在数据有限的场景中解决现有生物识别方法的局限性.
主要方法:
- 利用剩余神经网络 (ResNets) 来编码唇部的深度视频.
- 使用时间延迟神经网络 (TDNN) 架构用于编码语音信号.
- 集成的自我注意机制,用于有效的多式联接和耐噪声建模.
主要成果:
- 与基准数据集上的现有方法相比,实现了优异的性能.
- 与当前最先进的技术相比,平均性能提高了10%.
- 验证了模型在处理各种噪音条件中的有效性.
结论:
- 拟议的方法提供了一个高效和有效的解决方案,用于人身份识别在情景中,长时间的发言和RGB图像是不可行的.
- 该方法显示了在多式联络生物识别和其他相关领域更广泛的应用的巨大潜力.
相关概念视频
Facial Feedback Hypothesis
192
Charles Darwin proposed that facial expressions are an evolutionary adaptation for communication. He argued that these expressions are not influenced by culture but are universal across species. For example, a snarling expression with exposed teeth signals a threat in many animals, including humans. Darwin also suggested that displaying an emotion can intensify the feeling. Smiling, for example, could enhance one's sense of happiness. This idea laid the foundation for understanding the role...
192
Perceiving Loudness, Pitch, and Location
241
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
241
Force Classification
1.3K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
1.3K
Uniform Depth Channel Flow
98
Uniform depth channel flow keeps fluid depth consistent along channels such as irrigation canals. In natural channels, such as rivers, approximate uniform flow is often assumed. This condition occurs when the channel’s bottom slope matches the energy slope, balancing potential energy lost from gravity with head loss due to shear stress. This balance prevents depth changes along the channel length, resulting in a steady, uniform flow.Uniform flow in open channels with a constant cross-section...
98


