Prioritizing replay when future goals are unknown

Although hippocampal place cells replay nonlocal trajectories, the computational function of these events remains controversial. One hypothesis, formalized in a prominent reinforcement learning account, holds that replay plans routes to current goals. However, recent puzzling data appear to contradict this perspective by showing that replayed destinations lag current goals. These results may support an alternative hypothesis that replay updates route information to build a "cognitive map." Yet no similar theory exists to formalize this view, it is unclear how such a map is represented or what role replay plays in computing it. We address these gaps by introducing a theory of replay that learns a map of routes to candidate goals, before reward is available or when its location may change. Replay is then focused on current goals (as with planning) and/or potential future goals (like a map), depending on the animal's expectations about future goal switching. Our work thus generalizes the planning account to capture a general map-building function for replay, reconciling it with data, and revealing an unexpected relationship between the seemingly distinct hypotheses. The theory offers a unifying explanation why data from tasks with different goal dynamics have seemingly supported different hypotheses for the function of replay, and suggests new predictions for experiments testing these effects.