Dissociating language and thought in large language models.

Area of Science:

• Artificial Intelligence
• Computational Linguistics
• Cognitive Science

Background:

• Large language models (LLMs) approach human language mastery, but their capabilities are debated.
• Human language competence involves distinct formal (rule-based) and functional (contextual) aspects.
• Neuroscience reveals separate neural underpinnings for formal and functional linguistic competence in humans.

Purpose of the Study:

• To evaluate large language models (LLMs) based on the formal vs. functional linguistic competence distinction.
• To compare LLM performance on formal and functional tasks.
• To explore the implications for developing human-like AI language abilities.

Main Methods:

• LLMs were assessed on tasks designed to probe formal linguistic competence (e.g., grammar, syntax).
• LLMs were evaluated on tasks measuring functional linguistic competence (e.g., real-world language use, context understanding).
• Performance was analyzed in relation to human neuroscience findings on language processing.

Main Results:

• LLMs demonstrate significant proficiency in formal linguistic competence.
• LLM performance on functional linguistic competence tasks is inconsistent and often requires fine-tuning or external modules.
• A gap exists between LLMs' rule-based knowledge and their ability to use language effectively in context.

Conclusions:

• LLMs excel at mastering linguistic rules but lag in applying language functionally, mirroring human cognitive specializations.
• Achieving human-like language capabilities in AI may necessitate developing distinct mechanisms for formal and functional competence.
• Future LLM development should focus on bridging the gap in functional linguistic competence for more robust language understanding and generation.