Spatial heterogeneity, predator cognition, and the evolution of color polymorphism in virtual prey

Cryptically colored prey species are often polymorphic, occurring in multiple distinctive pattern variants. Visual predators promote such phenotypic variation through apostatic selection, in which they attack more abundant prey types disproportionately often. In heterogeneous environments, disruptive selection to match the coloration of disparate habitat patches could also produce polymorphism, but how apostatic and disruptive selection interact in these circumstances is unknown. Here we report the first controlled selection experiment on the evolution of prey coloration on heterogeneous backgrounds, in which blue jays (Cyanocitta cristata) searched for digital moths on mixtures of dark and light patches at three different scales of heterogeneity. As predicted by ecological theory, coarse-grained backgrounds produced a functional dimorphism of specialists on the two patch types; fine-grained backgrounds produced generalists. The searching strategies of the jays also varied with the habitat configuration, however. Complex backgrounds with many moth-like features elicited a slow, serial search that depended heavily on selective attention. The result was increased apostatic selection, producing a broad range of moth phenotypes. Backgrounds with larger, more uniform patches allowed the birds to focus on the currently most rewarding patch type and to search entire patches rapidly in parallel. The result was less apostatic selection and lower phenotypic variability. The evolution of polymorphism in camouflaged prey depends on a complex interaction between habitat structure and predator cognition.