Learning spatial invariance with the trace rule in nonuniform distributions

Convolutional models of object recognition achieve invariance to spatial transformations largely because of the use of a suitably defined pooling operator. This operator typically takes the form of a max or average function defined across units tuned to the same feature. As a model of the brain's ventral pathway, where computations are carried out by weighted synaptic connections, such pooling can lead to spatial invariance only if the weights that connect similarly tuned units to a given pooling unit are of approximately equal strengths. How identical weights can be learned in the face of nonuniformly distributed data remains unclear. In this letter, we show how various versions of the trace learning rule can help solve this problem. This allows us in turn to explain previously published results and make recommendations as to the optimal rule for invariance learning.