Visual short-term memory and motor planning

Despite extensive experimental work showing that memory for visual information in prior fixations is limited, the nature and content of the information maintained across fixations, when vision functions in its natural context, is not well determined. To gain insight into what memory representations might be needed to support vision in the natural world, we examined eye and hand movements while subjects made a sandwich and while they copied a toy model in a virtual environment. Patterns of eye-hand coordination and fixation sequences suggest the need for planning and coordinating movements over a period of a few seconds. Since the movement plan is initiated when the eye is in a different position from that when the movement itself is made, the planning must be in a coordinate frame that is independent of eye position. Movement planning thus requires a representation of the spatial structure in a scene that is built up over different fixations.