Aligning brains into a shared space improves their alignment with large language models

Area of Science:

• Neuroscience
• Computational Linguistics
• Machine Learning

Background:

• Large language models (LLMs) can predict neural activity during natural language processing (NLP).
• Prior methods evaluating LLM encoding models within individual participants limit generalizability.
• Electrocorticography (ECoG) provides high-resolution neural data for studying brain activity.

Purpose of the Study:

• To develop a more generalizable method for predicting neural activity using LLMs.
• To enhance the accuracy of LLM-based neural encoding models.
• To investigate cross-participant commonalities in neural responses to language.

Main Methods:

• Analyzed ECoG data from eight participants listening to the same podcast.
• Developed and applied a shared response model to estimate a common information space across participants.
• Evaluated LLM-based encoding model performance within individual and shared spaces.

Main Results:

• A shared response model significantly improved LLM-based neural encoding accuracy by 37% (from r=0.188 to r=0.257).
• The shared space enabled denoising of individual brain responses.
• The most substantial improvements were observed in language-specialized brain regions, including the superior temporal gyrus and inferior frontal gyrus.

Conclusions:

• Estimating a shared neural information space enhances the generalizability and accuracy of LLM-based encoding models.
• This approach offers a promising method for decoding brain activity related to language comprehension.
• Findings suggest that common neural representations of language exist across individuals.