Prediction as a basis for skilled reading: insights from modern language models

Area of Science:

• Cognitive Science
• Computational Linguistics
• Neuroscience

Background:

• Reading is a complex skill, not innate, with adults exhibiting remarkable reading speed.
• The cognitive mechanisms underlying efficient reading, particularly predictive processing, remain an active area of research.

Purpose of the Study:

• To investigate the role of upcoming word predictions in skilled reading behavior.
• To explore the relationship between predictions generated by a language model and human eye-movement patterns during reading.

Main Methods:

• Utilized the GPT-2 language model to generate predictions for words in text passages.
• Correlated language model predictions with fine-grained eye-movement data (e.g., fixation durations, word skipping) from adult readers.

Main Results:

• GPT-2 predictions accurately reflected upcoming words and correlated significantly with eye-movement behaviors.
• Prediction errors, where strong predictions were incorrect, led to measurable costs in reading fixation durations.
• Eye-movement patterns, including word skipping and fixation time, were demonstrably guided by predictive processing.

Conclusions:

• Predictive processing based on statistical text analysis plays a crucial role in guiding eye movements during reading.
• Modern language models can effectively model and inform our understanding of human language processing and reading mechanisms.
• These findings offer new insights into the predictive nature of reading and language comprehension.