The Plausibility of Sampling as an Algorithmic Theory of Sentence Processing

Area of Science:

• Computational Linguistics
• Cognitive Science
• Natural Language Processing

Background:

• Surprisal theory posits that word processing time correlates with contextual surprise.
• Existing incremental parsing algorithms do not directly predict this relationship.
• Previous models assumed linear scaling of processing cost with surprisal.

Purpose of the Study:

• To investigate algorithms whose runtime scales with surprisal.
• To test predictions of sampling-based algorithms against empirical reading time data.
• To contrast new predictions with existing surprisal theory.

Main Methods:

• Developed and analyzed simple sampling-based algorithms.
• Empirically studied the relationship between estimated surprisal and reading time.
• Utilized modern language models for surprisal estimation.

Main Results:

• Sampling algorithms predict superlinear increases in reading time with surprisal.
• These algorithms also predict increased variance in reading time.
• Empirical data confirms superlinear scaling and increased variance with improved language models.

Conclusions:

• Sampling-based algorithms offer a more accurate model of reading time and surprisal.
• Findings challenge the linear assumptions of traditional surprisal theory.
• Modern language models support the predictions of sampling-based computational models.