Text-to-music generation models capture musical semantic representations in the human brain.

Area of Science:

• Neuroscience
• Machine Learning
• Music Cognition

Background:

• Understanding how the brain processes complex sensory information like music is a key challenge in neuroscience.
• Machine learning models offer powerful tools for decoding brain activity and understanding neural representations.

Purpose of the Study:

• To investigate the functional correspondence between music-specific information and human brain activity.
• To explore the potential of machine learning models, specifically MusicLM, in reconstructing and understanding music perception.

Main Methods:

• Functional magnetic resonance imaging (fMRI) was used to measure brain activity in human subjects listening to music.
• Music reconstruction was performed by conditioning a MusicLM generator on brain data.
• Encoding modeling analysis was employed to assess the representation of semantic and textual music information in the brain.

Main Results:

• Generated music using MusicLM conditioned on brain data showed semantic similarities (genre, mood, instrumentation) to the original stimuli.
• Semantic information from music and its textual descriptions were found to be represented in overlapping regions around the auditory cortex.
• MusicLM features predicted brain activity in the primary auditory cortex more effectively than non-music-focused models.

Conclusions:

• Brain activity can be aligned with machine learning models to understand neural representations of music.
• MusicLM demonstrates a strong capability in capturing and predicting music-related brain activity, highlighting its relevance for studying auditory perception.
• The findings suggest that semantic aspects of music are processed in distributed neural networks centered around the auditory cortex.