在人类听觉皮层和反复的自动语音识别系统中对语言表示的平行等级编码
Menoua Keshishian1,2, Gavin Mischler1,2, Samuel Thomas3
1Department of Electrical Engineering, Columbia University, New York, NY, USA.
bioRxiv : the preprint server for biology
|February 20, 2025
概括
这项研究揭示了人类大脑和先进的人工智能语音系统都在层次上处理语言. 两者都编码了从声学到语义的类似特征,表明了语音处理中的共享计算原则.
科学领域:
- 神经科学是一个神经科学.
- 计算语言学 计算语言学
- 人工智能的人工智能
背景情况:
- 人类语音处理将声信号转化为语言表示,激发自动语音识别 (ASR).
- 以前的ASR-大脑比较受到了生物学上不可思议的模型,狭窄的特征和关注可预测性而不是共享表示的限制.
- 将大脑与基于文本的模型进行比较的研究忽略了关键的声学处理阶段.
研究的目的:
- 调查在语音处理过程中人类大脑和反复出现的ASR模型之间的共享等级表示.
- 通过使用高分辨率的内记录和先进的ASR模型来弥合先前研究的差距.
- 探索两种系统如何从声学到语义层次编码语言特征.
主要方法:
- 利用人类参与者的高分辨率内脑电图 (iEEG) 记录.
- 采用反复的人工智能模型进行自动语音识别.
- 将神经活动模式与ASR模型相应层中的表示进行了比较.
主要成果:
- 在大脑和ASR模型之间对语言特征的层次编码中展示了惊人的对应性.
- 显示了听觉皮层区域的神经活动与特定的ASR模型层之间的对齐.
- 证实两种系统在处理阶段都编码了相似的特征:声学,语音,词汇和语义.
结论:
- 人类大脑和ASR系统在语言处理方面采用了相似的策略,尽管存在结构上的差异.
- 结果提供了对语言表示的最佳计算原理的见解.
- 突出了塑造人类和人工语言处理的共同约束.
更多相关视频
相关概念视频
01:19Encoding
123
Information enters the brain through encoding, which is the input of information into the memory system. Once sensory information is received from the environment, the brain labels or codes it. The information is then organized with similar information and connected to existing concepts. Encoding occurs through automatic processing and effortful processing.
Automatic processing involves the encoding of details like time, space, frequency, and the meaning of words, usually done without conscious...
Automatic processing involves the encoding of details like time, space, frequency, and the meaning of words, usually done without conscious...
123
01:31Hearing
51.8K
When we hear a sound, our nervous system is detecting sound waves—pressure waves of mechanical energy traveling through a medium. The frequency of the wave is perceived as pitch, while the amplitude is perceived as loudness.
51.8K
01:15Auditory Pathway
4.6K
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
4.6K
01:13The Cochlea
44.5K
The cochlea is a coiled structure in the inner ear that contains hair cells—the sensory receptors of the auditory system. Sound waves are transmitted to the cochlea by small bones attached to the eardrum called the ossicles, which vibrate the oval window that leads to the inner ear. This causes fluid in the chambers of the cochlea to move, vibrating the basilar membrane.
44.5K
01:10Higher Mental Functions of the Brain: Language
715
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
715
01:21Perceiving Loudness, Pitch, and Location
188
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
188