Invariant template matching in systems with spatiotemporal coding: A matter of instability

We consider the design principles of algorithms that match templates to images subject to spatiotemporal encoding. Both templates and images are encoded as temporal sequences of samplings from spatial patterns. Matching is required to be tolerant to various combinations of image perturbations. These include ones that can be modeled as parameterized uncertainties such as image blur, luminance, and, as special cases, invariant transformation groups such as translation and rotations, as well as unmodeled uncertainties (noise). For a system to deal with such perturbations in an efficient way, they are to be handled through a minimal number of channels and by simple adaptation mechanisms. These normative requirements can be met within the mathematical framework of weakly attracting sets. We discuss explicit implementation of this principle in neural systems and show that it naturally explains a range of phenomena in biological vision, such as mental rotation, visual search, and the presence of multiple time scales in adaptation. We illustrate our results with an application to a realistic pattern recognition problem.