Mechanisms for handling nested dependencies in neural-network language models and humans.

Area of Science:

• Computational Linguistics
• Cognitive Neuroscience
• Artificial Intelligence

Background:

• Recursive processing is key to human language comprehension, but its neural basis is unclear.
• Deep learning models offer a potential avenue to investigate neural mechanisms of language processing.

Purpose of the Study:

• To investigate if deep learning models, specifically recurrent neural networks with Long Short-Term Memory units, can replicate human grammatical number and gender agreement in sentence processing.
• To compare model performance with human behavioral data on agreement tasks, especially those involving long-distance and embedded dependencies.

Main Methods:

• Trained a recurrent neural network with Long Short-Term Memory units on a large text corpus to predict the next word.
• Analyzed the network's internal representations for specialized units handling syntactic agreement.
• Conducted a behavioral experiment with human participants to detect number agreement violations in sentences with varying complexity.

Main Results:

• The artificial neural network developed specialized units that successfully managed local and long-distance syntactic agreement for grammatical number.
• Model and human error patterns showed significant similarities in agreement violation detection.
• Humans outperformed the model on sentences with embedded long-range dependencies, remaining above chance while the model's performance dropped below chance.

Conclusions:

• Deep learning models can partially mimic human sentence processing, offering testable hypotheses about linguistic mechanisms.
• While effective for basic agreement, current models do not fully capture human capacity for complex recursion.
• Comparing artificial and human linguistic processing provides valuable insights into both AI and cognitive science.