Temporal prediction captures retinal spiking responses across animal species

The retina's role in visual processing has been viewed as two extremes: an efficient compressor of incoming visual stimuli, akin to a camera, or a predictor of future stimuli. Addressing this dichotomy, we developed a spiking neural network model of the retina trained on natural movies under metabolic-like constraints to either encode the present or to predict future scenes. When optimized for efficient temporal prediction ~100 ms into the future, the model not only captures retina-like receptive fields and their mosaic-like organizations, but also exhibits complex retinal processes such as latency coding, motion anticipation, differential motion tuning, and stimulus-omission responses. Notably, the temporal prediction model also accurately predicts the way retinal ganglion cells respond across different animal species to natural images and movies. Our findings suggest that the retina is not merely a compressor of visual input, but rather is fundamentally organized to provide the brain with foresight into the visual world.