Extreme synergy: spatiotemporal correlations enable rapid image reconstruction from computer-generated spike trains

Over the brief time intervals available for processing retinal output, the number of spikes generated by individual ganglion cells can be quite variable. Here, two examples of extreme synergy are used to illustrate how realistic long-range spatiotemporal correlations can greatly improve the quality of retinal images reconstructed from computer-generated spike trains that are 25-400 ms in duration, approximately the time between saccadic eye movements. Firing probabilities were specified both explicitly: using time-varying waveforms consistent with stimulus-evoked oscillations measured experimentally, and implicitly: by superimposing realistic fixational eye movements on a biophysical model of primate outer retina. Synergistic encoding was investigated across arrays of model neurons up to 32 x 32 in extent, containing over 1 million pairwise correlations. The difficulty of estimating pairwise, spatiotemporal correlations on single trials from only a few events was overcome by using oscillatory, local multiunit activity to weight contributions from all spike pairs. Stimuli were reconstructed using either an independent rate code or the first principal component of the single-trial, pairwise correlation matrix. Spatiotemporal correlations mediated dramatic improvements in signal/noise without eliminating fine spatial detail, demonstrating how extreme synergy can support rapid image reconstruction using far fewer spikes than required by an independent rate code.