Are language models models?

Area of Science:

• Cognitive Science
• Artificial Intelligence
• Computational Neuroscience

Background:

• The utility of language models (LMs) as scientific model systems has been proposed.
• Assessing LMs requires a multi-level framework, such as Marr's levels of analysis.

Purpose of the Study:

• To critically evaluate the claim that language models serve as model systems.
• To analyze language models through the lens of Marr's three levels: computational theory, algorithm/representation, and physical implementation.

Main Methods:

• A theoretical assessment of language models was conducted.
• The evaluation framework comprised Marr's three levels of analysis: computational theory, algorithm/representation, and implementation.

Main Results:

• Language models fail to meet the criteria for cognitive models at the implementation level.
• The claim is poorly motivated at the algorithmic-representational level.
• The computational theory level presents significant challenges for language models as cognitive models.

Conclusions:

• Language models are better understood as sophisticated tools rather than cognitive models.
• Overstating the cognitive capabilities of LMs contributes to unwarranted hype in the field of artificial intelligence.