最佳运输和对比学习为大脑解码音乐感知
概括
这项研究使用功能磁共振成像 (fMRI) 来增强音乐从大脑活动解码. 结合最佳传输和对比学习的新框架提高了从神经数据中重建音乐的准确性.
科学领域:
- 神经科学是一个神经科学.
- 机器学习 机器学习
- 信号处理 信号处理
背景情况:
- 大脑解码旨在从大脑活动中重建外部刺激.
- 使用功能磁共振成像 (fMRI) 解码音乐,由于复杂的听觉处理和fMRI的时间限制,存在挑战.
研究的目的:
- 引入一种新的解码框架,以从fMRI数据中改进音乐重建.
- 通过使用预训练的多式模式 (CLAP) 增强fMRI活动和潜在的音乐表现之间的对齐.
主要方法:
- 建议采用一种结合最佳运输和对比学习的双损失方法.
- 最佳运输确保了大脑预测和真实音乐嵌入之间的结构一致性.
- 相反的学习完善了嵌入空间,以获得更好的特征映射和检索准确性.
主要成果:
- 这种新方法在5名受试者的fMRI数据上提高了解码性能.
- 顶级-1准确度从22.1%增加到29.3%,超过了传统的基于回归的方法.
- 该框架成功地解码了GTZAN数据集中的音乐曲目.
结论:
- 整合最佳传输和对比学习显著改善了大脑解码性能.
- 这种方法显示出大脑与计算机接口 (BCI) 和其他感官领域的应用潜力.
- 这些发现可能有助于理解听觉处理障碍和神经康复策略.
相关概念视频
Auditory Perception
998
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
998
Perceiving Loudness, Pitch, and Location
896
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
896
Parallel Processing
605
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
605


