Brains and algorithms partially converge in natural language processing.

Area of Science:

• Neuroscience
• Computational Linguistics
• Artificial Intelligence

Background:

• Deep learning models exhibit brain-like activations when predicting text.
• The computational drivers of this similarity are not well understood.
• Understanding these drivers can illuminate natural language processing in both artificial and biological systems.

Purpose of the Study:

• To systematically compare various deep language models.
• To identify computational principles underlying brain-like sentence representations.
• To map algorithm activations onto human brain responses.

Main Methods:

• Analysis of brain responses (fMRI and MEG) to 400 sentences in 102 subjects.
• Systematic comparison of diverse deep language models.
• Correlation of model architecture, training, and performance with brain representations.

Main Results:

• Model similarity to brain activity is primarily driven by word prediction from context.
• This similarity highlights the development of perceptual, lexical, and compositional representations in cortical regions.
• Specific model features independently explain the generation of brain-like representations.

Conclusions:

• Modern language algorithms partially converge on brain-like processing solutions.
• Word prediction from context is a key factor in generating neural representations.
• This research offers a framework for understanding the foundations of natural language processing.