Learning continuous updating and control of neural population activities from sparse time samples

The brain frequently updates or controls its internal representations of sensory or motor variables over time. An example is transsaccadic updating of stimuli's retinotopic positions: When the eye moves in one direction, the positions are updated continuously in the opposite direction, and the process is controlled by the corollary discharge (CD) of the saccade motor command. We previously trained neural networks to perform such updating using the desired time courses of stimuli's positions at every time step. However, this may not be biologically plausible because, before a system learns to perform the task, it may not have access to complete time courses and the actual visual inputs are delayed and thus incorrect. Here we show that by adding the continuity equation to the loss, we can train neural networks with only stimuli's pre- and post-saccadic retinotopic positions (without specifying the intermediate transitions), and with heavily down-sampled time courses. Moreover, both the connectivity patterns and the CD control signal can be learned simultaneously. Since the continuity equation can be applied to waves of neural population activities that are distributed representations of variables, our work also suggests an advantage of distributed representations which are ubiquitous in the brain.