AVbook,一个高率的叙事视听演讲集体,用于调查多模式语音感知
Enrico Varano1, Pierre Guilleminot1, Tobias Reichenbach2
1Department of Bioengineering and Centre for Neurotechnology, Imperial College London, London, United Kingdom.
The Journal of the Acoustical Society of America
|May 30, 2023
概括
这项研究介绍了AVbook,这是一个用于认知神经科学的新视听语音语文库. AVbook提供高质量的录音,以推进视听语音处理和识别方面的研究.
科学领域:
- 认知神经科学 认知神经科学
- 语音处理 语音处理
- 计算机视觉 计算机视觉
背景情况:
- 视觉线索显著提高语音理解,特别是在杂的环境中.
- 现有的视听语音体缺乏足够的持续时间和质量来进行先进的神经生物学研究.
- 持续的自然语言对于理解视听一体化机制至关重要.
研究的目的:
- 为认知神经科学研究开发一个高质量的视听语音语文库 (AVbook).
- 支持开发用于视听语音识别的算法.
- 促进对视听语音处理的神经生物学研究.
主要方法:
- 创建了AVbook:来自两个演讲者阅读叙事文本的3.6小时的视听录音.
- 在高率 (119.88 fps) 获取录音.
- 包括电话水平对齐,注意力测试问题,摘要和同步标记.
主要成果:
- 开发了AVbook,这是一个全面的视听语音集体.
- 在试点研究中验证了注意力测试问题.
- 在实验设置中确保体的适合同步.
结论:
- AVbook解决了对扩展,高质量的视听语音数据的需求.
- 该语料库是公开的,以推进视听语言研究.
- 促进神经科学和人工智能领域的跨学科研究.
相关概念视频
Auditory Perception
389
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
389
Perception of Sound Waves
4.5K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
4.5K
Perceiving Loudness, Pitch, and Location
278
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
278
Chunking and Rehearsal in Sensory Memory
253
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
253
Higher Mental Functions of the Brain: Language
940
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
940


