Predicting responses of nonlinear neurons in monkey striate cortex to complex patterns

The overwhelming majority of neurons in primate visual cortex are nonlinear. For those cells, the techniques of linear system analysis, used with some success to model retinal ganglion cells and striate simple cells, are of limited applicability. As a start toward understanding the properties of nonlinear visual neurons, we have recorded responses of striate complex cells to hundreds of images, including both simple stimuli (bars and sinusoids) as well as complex stimuli (random textures and 3-D shaded surfaces). The latter set tended to give the strongest response. We created a neural network model for each neuron using an iterative optimization algorithm. The recorded responses to some stimulus patterns (the training set) were used to create the model, while responses to other patterns were reserved for testing the networks. The networks predicted recorded responses to training set patterns with a median correlation of 0.95. They were able to predict responses to test stimuli not in the training set with a correlation of 0.78 overall, and a correlation of 0.65 for complex stimuli considered alone. Thus, they were able to capture much of the input/output transfer function of the neurons, even for complex patterns. Examining connection strengths within each network, different parts of the network appeared to handle information at different spatial scales. To gain further insights, the network models were inverted to construct "optimal" stimuli for each cell, and their receptive fields were mapped with high-resolution spots. The receptive field properties of complex cells could not be reduced to any simpler mathematical formulation than the network models themselves.