Fundamental computational constraints on the time course of perception and action

A cognitive system faced with contingent events that cause rapid changes in sensory data may (i) incrementally incorporate new data into the ongoing perceptual and motor processing; or (ii) restart processing on each new event; or (iii) sample the data and hold onto the sample until its processing is complete, while disregarding any contingent changes. We offer a set of computational first-principles arguments for a hypothesis, according to which any system that contends with certain classes of perception and behavioral control tasks must include the sample-and-hold option (possibly alongside the other two, which may be useful in other tasks). This hypothesis has implications for understanding the dynamics of perception and action. In particular, a sample-and-hold channel necessarily processes sensory data on some kind of cycle (which does not imply precise periodicity). Further, being prepared to face the world at all times requires that the sampling that initiates each cycle be triggered by every significant action on part of the agent itself, such as saccades. We survey a range of evidence for the sample-and-hold functionality, touching upon diverse phenomena such as attentional blink and backward masking, the yoking of olfaction to respiration, thalamocortical interactions, and metastable brain dynamics in perception and consciousness.