Divergences between Language Models and Human Brains

Area of Science:

• Cognitive Science
• Neuroscience
• Artificial Intelligence

Background:

• Recent studies suggest language models (LMs) and human brains share computational principles in language processing.
• However, significant differences exist in how LMs and humans represent and utilize language.

Purpose of the Study:

• To systematically explore divergences between human and machine language processing.
• To identify specific domains where LMs fall short compared to human cognition.

Main Methods:

• Examined differences between LM representations and human brain responses using Magnetoencephalography (MEG).
• Analyzed two datasets of subjects reading and listening to narrative stories.
• Employed a large language model (LLM)-based data-driven approach.

Main Results:

• Identified social/emotional intelligence and physical commonsense as domains poorly captured by current LMs.
• Validated these findings through human behavioral experiments.
• Observed that fine-tuning LMs on these specific domains enhances their alignment with human brain activity.

Conclusions:

• The gap in LM performance is likely due to insufficient representation of social/emotional and physical knowledge.
• Fine-tuning LMs can bridge the gap between machine and human language processing in these critical domains.