Deep Artificial Neural Networks Reveal a Distributed Cortical Network Encoding Propositional Sentence-Level Meaning.

Area of Science:

• Cognitive Neuroscience
• Computational Linguistics
• Artificial Intelligence

Background:

• Understanding sentence-level meaning construction in the brain is a key challenge.
• Previous models used word co-occurrence (bag-of-words) to map semantic representations but neglected sentence structure.
• It's unclear if brain activation reflects unified sentence meaning or individual word meanings.

Purpose of the Study:

• To investigate how the brain encodes unified sentence-level meaning.
• To compare deep neural network models with traditional bag-of-words and rule-based models in predicting brain activity.
• To determine if sentence meaning is represented in a distributed network or localized sites.

Main Methods:

• Functional magnetic resonance imaging (fMRI) was used to record brain activity in 14 participants reading 240 sentences.
• A recurrent deep artificial neural network (InferSent) was employed to generate propositional sentence representations.
• Voxelwise encoding modeling was used to predict fMRI activation patterns.

Main Results:

• The InferSent model predicted fMRI activation significantly better than bag-of-words or rule-based models.
• This predictive power was observed across a distributed brain network spanning temporal, parietal, and frontal cortex.
• Findings suggest that propositional sentence-level meaning is represented throughout this network.

Conclusions:

• Deep neural networks capturing sentence structure are crucial for understanding brain's semantic processing.
• The brain appears to represent unified sentence meaning across multiple cortical regions.
• This study provides evidence for distributed semantic representations beyond individual word meanings.