To model human linguistic prediction, make LLMs less superhuman.

Area of Science:

• Cognitive Science
• Computational Linguistics
• Artificial Intelligence

Background:

• Humans make predictions about upcoming words during reading, influencing behavior.
• Large language models (LLMs) also predict words, leading to their use as models of human linguistic prediction.

Purpose of the Study:

• To investigate the declining ability of LLMs to explain human reading behavior.
• To identify the reasons behind the misalignment between LLM predictions and human reading patterns.

Main Methods:

• Comparative analysis of LLM word prediction capabilities versus human reading behavior.
• Examination of LLM memory (short-term and long-term) and training data in relation to predictive accuracy.

Main Results:

• LLMs' word prediction accuracy has improved significantly.
• Despite improved prediction, LLMs' explanatory power for human reading behavior has decreased.
• LLMs exhibit 'superhuman' predictive abilities due to extensive training data and superior memory recall.

Conclusions:

• Current LLMs are too "superhuman" to effectively model human reading.
• Future research should focus on developing LLMs with human-like memory capacities.
• New experimental paradigms are needed to measure human-LLM alignment in linguistic prediction.