Derivational morphology reveals analogical generalization in large language models.

Area of Science:

• Computational Linguistics
• Cognitive Science
• Artificial Intelligence

Background:

• Investigating linguistic generalization in large language models (LLMs) is crucial for understanding their capabilities.
• Prior research often focused on regular linguistic phenomena, making it difficult to distinguish between rule-based and analogical learning.
• The role of analogical reasoning in LLM linguistic generalization remains under-explored.

Purpose of the Study:

• To determine whether linguistic generalization in LLMs is driven by rules or analogy.
• To examine English adjective nominalization, a variable linguistic phenomenon, to differentiate between learning mechanisms.
• To introduce a novel method for analyzing LLM generalization by comparing cognitive models with LLM predictions.

Main Methods:

• Focused on the GPT-J model to investigate linguistic generalization.
• Developed and fitted cognitive models of rule-based and analogical learning to LLM training data.
• Compared model predictions with GPT-J's predictions on nonce adjectives, particularly those with variable nominalization patterns.

Main Results:

• Both rule-based and analogical models equally explained GPT-J's predictions for regular adjective nominalization.
• The analogical model significantly outperformed the rule-based model for variable nominalization patterns.
• GPT-J's sensitivity to word frequencies, even for regular forms, aligns with an analogical account.

Conclusions:

• The findings refute the hypothesis that GPT-J uses rules for linguistic generalization in adjective nominalization.
• Evidence strongly suggests that analogical processes are the primary mechanism underlying GPT-J's linguistic generalization.
• This study highlights the significant role of analogy in the linguistic capabilities of large language models.