文本到音乐生成模型捕捉了人类大脑中的音乐语义表示
Timo I Denk1, Yu Takagi2,3,4, Takuya Matsuyama5,6
1Google DeepMind, 1600 Amphitheatre Parkway Mountain View, Mountain View, CA, USA. simssosdenk@gmail.com.
Nature communications
|November 29, 2025
概括
研究人员将大脑活动与机器学习模型对齐,以理解音乐的表现. 音乐LM准确地预测了大脑活动,显示语义音乐信息在听觉皮层附近被处理.
科学领域:
- 神经科学是一个神经科学.
- 机器学习 机器学习
- 音乐认知 音乐认知
背景情况:
- 了解大脑如何处理音乐等复杂的感官信息是神经科学的一个关键挑战.
- 机器学习模型为解码大脑活动和理解神经表示提供了强大的工具.
研究的目的:
- 研究音乐特定信息与人类大脑活动之间的功能对应.
- 探索机器学习模型的潜力,特别是MusicLM,用于重建和理解音乐感知.
主要方法:
- 功能磁共振成像 (fMRI) 用于测量听音乐的人类受试者的大脑活动.
- 通过将MusicLM发电机调节到大脑数据上来进行音乐重建.
- 用编码建模分析来评估大脑中语义和文本音乐信息的表示.
主要成果:
- 使用 MusicLM 以大脑数据为条件生成的音乐显示了与原始刺激的语义相似性 (类型,情绪,乐器).
- 发现来自音乐和其文本描述的语义信息在听觉皮层周围的重叠区域中呈现.
- 音乐LM功能预测了主要听觉皮层中的大脑活动,比非音乐专注的模型更有效.
结论:
- 大脑活动可以与机器学习模型保持一致,以理解音乐的神经表征.
- 音乐LM在捕捉和预测与音乐相关的大脑活动方面表现出强大的能力,突出了它对研究听觉感知的相关性.
- 这些发现表明,音乐的语义方面在以听觉皮层为中心的分布式神经网络中被处理.
更多相关视频
13:12Translational Brain Mapping at the University of Rochester Medical Center: Preserving the Mind Through Personalized Brain Mapping
Published on: August 12, 2019
46.3K
07:52Author Spotlight: Investigating Vocal Information Representation in Small Primates and Its Alteration by Psychiatric Disorders Using Noninvasive EEG
Published on: July 26, 2024
1.3K
相关概念视频
Brain Waves
3.8K
Brain waves are electrical signals generated by the neurons in the brain, which are regularly monitored to measure mental activities. Brain waves and their frequency ranges can be measured using an electroencephalogram or EEG. There are four main types of brain waves, each with distinct characteristics:
3.8K
Hearing
56.3K
When we hear a sound, our nervous system is detecting sound waves—pressure waves of mechanical energy traveling through a medium. The frequency of the wave is perceived as pitch, while the amplitude is perceived as loudness.
56.3K
Perception of Sound Waves
5.4K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
5.4K
Encoding
724
Information enters the brain through encoding, which is the input of information into the memory system. Once sensory information is received from the environment, the brain labels or codes it. The information is then organized with similar information and connected to existing concepts. Encoding occurs through automatic processing and effortful processing.
Automatic processing involves the encoding of details like time, space, frequency, and the meaning of words, usually done without conscious...
Automatic processing involves the encoding of details like time, space, frequency, and the meaning of words, usually done without conscious...
724
Perceiving Loudness, Pitch, and Location
896
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
896
Concepts and Prototypes
480
The human nervous system handles vast amounts of information by translating sensory stimuli into neural impulses, which the brain processes, creating thoughts expressed through language or stored as memories. The brain also synthesizes information from emotions and memories, which significantly influence thoughts and behaviors. This intricate process creates a comprehensive mental picture.
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
The brain organizes this information using concepts, which are mental categories grouping linguistic data,...
480
