Conflict between representational formats and procedures constrains visual tracking

Computer-assisted design software, such as PowerPoint, groups separate elements to create structured objects whose parts can be manipulated relative to the whole. Might visual perception perform a similar operation, grouping scene elements and representing them with internal reference to the emergent structure? The proposal is central yet perennially controversial in cognitive neuroscience and philosophy of mind. In a series of seven preregistered tracking experiments, we identify internally referenced representations in human vision by relying on the logic that, as in computer-assisted design, a representation will make certain procedures easy to access while limiting others. Participants tracked a target among identical nontargets organized into geometric shapes. When the shapes rotated, performance was high, suggesting the use of internally referenced, shape-centered representations that naturally support rotational transformations. In contrast, performance declined in translation conditions, where constituent items moved independently, revealing a mismatch between the internally structured representation and the requisite, externally oriented tracking procedure. This effect replicated across diverse stimuli, including occluded, rounded, and irregular shapes, and was reduced when structure-inducing cues were disrupted. These findings identify representational format and its compatibility with task procedures as an important determinant of visual performance.