Shared computational principles for language processing in humans and deep language models.

Area of Science:

• Neuroscience
• Computational Linguistics
• Artificial Intelligence

Background:

• Traditional linguistic models are being challenged by deep learning advances.
• Autoregressive deep language models (DLMs) predict linguistic responses using next-word prediction.
• Understanding the neural basis of language processing is a key challenge.

Purpose of the Study:

• To investigate shared computational principles between the human brain and autoregressive DLMs during natural language processing.
• To provide empirical evidence for functional similarities in how humans and DLMs process narrative.

Main Methods:

• Electrocorticography (ECoG) was used to record brain responses from nine participants listening to a 30-minute podcast.
• Neural activity was analyzed in relation to the narrative's linguistic content.
• Computational analyses compared brain responses to the predictions and surprise signals generated by autoregressive DLMs.

Main Results:

• The human brain, like autoregressive DLMs, engages in continuous next-word prediction before word onset.
• Both the brain and DLMs match pre-onset predictions with incoming words to compute post-onset surprise.
• Contextual embeddings are crucial for word representation in both human language processing and DLMs.

Conclusions:

• Human language processing and autoregressive DLMs share fundamental computational principles, including prediction and surprise.
• Autoregressive DLMs offer a biologically feasible computational framework for studying the neural basis of language.
• This research bridges computational neuroscience and artificial intelligence in the domain of language.