Sentence processing by humans and machines: Large language models as a tool to better understand human reading

Area of Science:

• Cognitive Science
• Computational Linguistics
• Artificial Intelligence

Background:

• Understanding incremental language processing in humans is key to reading research.
• Quantifying the influence of word context on processing has been challenging.
• Advances in AI have led to sophisticated large language models (LLMs) capable of humanlike text generation.

Purpose of the Study:

• To review empirical findings on using LLMs to measure word processing load.
• To assess whether LLM-derived metrics predict human reading comprehension.
• To evaluate the methodological and theoretical implications of this approach.

Main Methods:

• Utilizing LLMs to estimate next-word probabilities based on linguistic context.
• Calculating information-theoretic metrics like entropy and surprisal from these probabilities.
• Analyzing the predictive power of these metrics on online human reading measures (e.g., eye-tracking, ERP).

Main Results:

• LLMs can model probabilistic relationships between words in a language.
• Entropy and surprisal derived from LLMs serve as quantitative measures of processing load.
• These metrics show potential in predicting variance in human reading comprehension data.

Conclusions:

• LLM-derived metrics offer a novel approach to studying incremental language processing.
• This methodology provides insights into the cognitive mechanisms of reading.
• Further research is needed to refine and validate these computational measures in cognitive science.