相关实验视频

Updated: Jul 2, 2025

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

对真实面孔和通过深度神经网络或面部动作编码系统生成的合成面孔的语音噪音感知的影响

Yingjia Yu¹, Anastasia Lado¹, Yue Zhang²

¹Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania.

bioRxiv : the preprint server for biology

|February 19, 2024

概括

合成的说话面孔可以提高语音感知,但当前的方法,如深度神经网络 (DNN) 和面部行动编码系统 (FACS) 提供有限的好处. 提高现实主义,特别是对于特定的语音,可以增强沟通.

科学领域:

语音感知语音感知
计算机视觉计算机视觉计算机视觉
人与计算机的互动.

背景情况:

合成说话面孔技术正在迅速发展,影响商业和学术领域.
虽然真实面孔增强了语音在噪音中的感知,但以前的合成面孔 (DNN) 仅提供了一半的好处.
面部动作编码系统 (FACS) 为更现实的合成面部生成提供了一个潜在的方法.

研究的目的:

通过深度神经网络 (DNN) 生成的合成面部对面部行动编码系统 (FACS) 的语音感知效益进行比较.
调查基于FACS的合成面部是否比基于DNN的合成面部提供更大的感知效益.
识别影响语音感知的合成面部生成的局限性.

主要方法:

人类观察者与视觉刺激识别了语音中的噪音:空白屏幕,真实脸,DNN合成脸或FACS合成脸.
在整体和语音层面分析了表现.
对合成面部的视觉特征进行了检查,以寻找特定的语音.

主要成果:

真正的面孔为语音噪音感知提供了最大的好处,复制了先前的发现.
无论是DNN还是FACS合成面孔都改善了感知,但比真实面孔的程度要小.
合成面孔在染涉及牙和嘴唇的音符方面表现出特殊的弱点 (例如, /f/, /v/, /th/).

结论:

目前的合成说话面孔技术 (DNN和FACS) 并不能完全复制真实面孔的感知优势.
在合成面部中提高特定音符可视化的现实性,可以大大提高它们的沟通效率.
这项研究为开发更有感知益处的合成说话面孔提供了路线图.

更多相关视频

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Published on: August 9, 2024

Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues

Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues

Published on: June 3, 2013

相关概念视频

Factors Affecting Perception

Factors Affecting Perception

Perception is influenced by perceptual set, context, motivation, and emotion. Perceptual set, or perceptual expectancy, refers to the tendency to perceive things in a particular way, influenced by previous experiences and expectations. This phenomenon affects the interpretation of stimuli, creating a set of mental tendencies and assumptions that impact sensory perceptions of sound, taste, touch, and sight.
An illustrative example of a perceptual set is the scenario where an airline pilot told...

Perceiving Loudness, Pitch, and Location

Perceiving Loudness, Pitch, and Location

The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...

Facial Feedback Hypothesis

Facial Feedback Hypothesis

Charles Darwin proposed that facial expressions are an evolutionary adaptation for communication. He argued that these expressions are not influenced by culture but are universal across species. For example, a snarling expression with exposed teeth signals a threat in many animals, including humans. Darwin also suggested that displaying an emotion can intensify the feeling. Smiling, for example, could enhance one's sense of happiness. This idea laid the foundation for understanding the role...