Using reinforcement learning to understand the emergence of "intelligent" eye-movement behavior during reading

The eye movements of skilled readers are typically very regular (K. Rayner, 1998). This regularity may arise as a result of the perceptual, cognitive, and motor limitations of the reader (e.g., limited visual acuity) and the inherent constraints of the task (e.g., identifying the words in their correct order). To examine this hypothesis, reinforcement learning was used to allow an artificial "agent" to learn to move its eyes to read as efficiently as possible. The resulting patterns of simulated eye movements resembled those of skilled readers and suggest that important aspects of eye-movement behavior might emerge as a consequence of satisfying the constraints that are imposed on readers. These results also suggest novel interpretations of some contentious empirical results, such as the fixation duration costs associated with word skipping (R. Kliegl & R. Engbert, 2005), and theoretical assumptions, for example the familiarity check in the E-Z Reader model of eye-movement control (E. D. Reichle, A. Pollatsek, D. L. Fisher, & K. Rayner, 1998).