Gaze reinstatement during working memory for natural scenes

The information we hold in mind with working memory (WM) may propagate beyond the cortex and out to the peripheral motor system. For instance, directionally biased hand and eye movements can express simple visuo-spatial features during WM maintenance. If such sensorimotor inflections are a form of WM representation, they may also express more detailed and task-adaptive information about WM content. Here, we ask whether gaze patterns carry item-specific detail about complex stimuli held in mind, and whether gaze during WM maintenance adapts to upcoming behavioral demands. To do this, we tracked male and female human eye movements during WM for naturalistic images, and we manipulated which image properties were most task-relevant (visual or semantic). In two experiments, we found that the eyes moved in item-specific spatiotemporal sequences during WM maintenance. Delay period gaze patterns retraced the scanpaths from stimulus encoding, echoing the 'gaze reinstatement' that has been observed during long-term memory retrieval. Therefore, gaze patterns during WM may reinstate visual encoding processes for maintenance. We also used time-resolved representational similarity analyses, guided by convolutional neural network modeling, to test the specificity of WM gaze patterns across a trial. We found that gaze representations were more distinct when the task prioritized visual detail, and they also ramped up in anticipation of the WM probe. Therefore, oculomotor WM signals are malleable to when and how the content will be used. WM may recruit the most primary structures in sensorimotor processing, where eye movements reflect flexible and future-oriented WM content information.Significance Statement The eyes are an outwardly accessible extension of the brain. They filter sensory input and enact motor output, but may also offer a rich window into underlying cognition. For instance, pupil dilation can reflect changing cognitive load, and biased eye movements can reflect basic spatial information about content held in mind with working memory (WM). WM is a critical neuro-cognitive function, but its physiological bases remain under debate. Here, we show that during WM the eyes can move in intricate patterns that read out the complex, real-world stimuli held in mind. This work shows how cognitive states influence oculomotor functions, and also advances theories of WM maintenance to include the most basic structures in sensorimotor processing.