Natural scene statistics and nonlinear neural interactions between frequency-selective mechanisms

Linear filtering is a basic concept in neural models of early sensory information processing. In particular the visual system has been described to perform a wavelet-like multi-channel decomposition by a set of independent spatial-frequency selective filter mechanisms. Here we suggest that this principle of linear filtering deserves a critical re-evaluation. We propose that an optimal adaptation to natural scene statistics would require AND-like nonlinear interactions between the frequency-selective filter channels. We describe how this hypothesis can be tested by predicted violations of the principle of linearity that should be observable if cortical neurons would actually implement the proposed nonlinearities. We further explain why these effects might have been easily overlooked in earlier tests of the linearity of neurons in primary visual cortex.