Carving the cognitive niche: optimal learning strategies in homogeneous and heterogeneous environments

A model learning system is constructed, in which an organism samples behaviors from a behavioral repertoire in response to a stimulus and selects the behavior with the highest payoff. The stimulus and most rewarding behavior may be kept in the organism's long-term memory and reused if the stimulus is encountered again. The value of the memory depends on the reliability of the stimulus, that is, how the corresponding payoffs of behaviors change over time. We describe how the inclusion of memory can increase the optimal sampling size in environments with some stimulus reliability. In addition to using memory to guide behavior, our organism may use information in its memory to choose the stimulus to which it reacts. This choice is influenced by both the organism's memory state and how many stimuli the organism can observe (its sensory capability). The number of sampled behaviors, memory length, and sensory capability are the variables that define the learning strategy. When all stimuli have the same reliability, there appears to be only a single optimal learning strategy. However, when there is heterogeneity in stimulus reliability, multiple locally optimal strategies may exist.