Low-Rank Tensor Encoding Models Decompose Natural Speech Comprehension Processes

Area of Science:

• Neuroscience
• Computational Linguistics
• Machine Learning

Background:

• Human language comprehension involves hierarchical processing across brain regions over time.
• Previous studies were limited by controlled settings, offering a coarse view of brain dynamics.
• Interpretable methods linking large language models (LLMs) to neural language processing are scarce.

Purpose of the Study:

• To develop an interpretable method for analyzing LLM encoding models in relation to brain activity during natural language processing.
• To characterize semantic information and temporal dynamics in neural signals using a novel decomposition technique.
• To improve the understanding of brain mechanisms underlying naturalistic language comprehension.

Main Methods:

• Developed a low-rank tensor regression method to decompose LLM encoding models.
• Applied the method to Magnetoencephalography (MEG) data from subjects listening to narrative stories.
• Compared the proposed model's performance against standard ridge regression encoding models.

Main Results:

• The low-rank tensor regression model demonstrated improved encoding performance with fewer components compared to standard methods.
• The method successfully decomposed LLM encoding models into interpretable components of semantics, time, and brain region activation.
• Identified diverse, interpretable neural response components sensitive to low-level and semantic language features, outperforming models controlled for basic audio and sentence features.

Conclusions:

• Low-rank tensor encoding models offer a valuable inductive bias for language encoding, improving performance and interpretability.
• The developed method effectively separates distinct language processing features within neural signals.
• This approach provides a powerful tool for uncovering complex language processes in naturalistic brain activity.