A TALE OF TWO TAILS: PREFERRED AND ANTI-PREFERRED NATURAL STIMULI IN VISUAL CORTEX

An ongoing quest in neuroscience is to find the preferred stimulus of a sensory neuron. This search lays the foundation for understanding how selectivity emerges in the primate visual stream-from simple edge-detecting neurons to highly-selective face neurons-as well as for the architectures and activation functions of deep neural networks. The prevailing notion is that a visual neuron primarily responds to a single preferred visual feature, like an oriented edge or the shape of an object, resulting in a "one-tailed" distribution of responses to natural images. However, surprisingly, we instead find "two-tailed" response distributions of primate visual cortical neurons, suggesting that these neurons have both preferred and anti-preferred stimuli. We experimentally validated anti-preferred stimuli by recording responses from macaque V4 to model-optimized stimuli. We find that these anti-preferred stimuli are important for describing a neuron's tuning, as both preferred and anti-preferred images are needed to predict a neuron's responses to natural images. Moreover, in a psychophysics task, humans rely on anti-preferred images to interpret and predict V4 stimulus tuning; this was not the case for internal units from a deep neural network. Interestingly, we find no discernible differences in image statistics between preferred and anti-preferred images. This suggests that by encoding anti-preferred features, a V4 population seemingly doubles its capacity for feature selectivity, allowing for a more flexible downstream readout. Overall, we establish anti-preferred stimuli as an important encoding property of V4 neurons. Our work embarks on a new quest in neuroscience to search for anti-preferred stimuli along the visual stream and offers a new perspective on how feature selectivity arises in the visual cortex and deep neural networks.