Eligibility-trace-gated depression in predecessor feature learning enables post-reward exploration

Spatial navigation and exploration require flexible learning mechanisms that can adapt to changing environmental demands. While dopamine's role in reward prediction error is well-established, the computational functions of other neuromodulators in reinforcement learning remain less understood. Here, we investigate how acetylcholine (ACh) modulation affects predecessor feature (PF)-based learning, a computational framework that combines successor representations with eligibility traces for retrospective credit assignment. We developed an ACh-modulated PF algorithm (ACh-PF) implementing synaptic depression via eligibility-trace outer products ([Formula: see text]), hypothesized to promote exploration by attenuating reinforcement of recently traversed transitions. Using n-arm radial mazes, we compared conventional navigation (episodes terminate at reward) with a post-reward exploration criterion requiring visits to all arm endpoints. In conventional mode, all agents achieved near-optimal performance. Under the post-reward criterion, the PF baseline largely failed, whereas ACh-PF exhibited a non-monotonic dependence on [Formula: see text]: performance improved sharply within a narrow intermediate regime and degraded at higher gains, consistent with a knee-like transition followed by over-depression. The effective window narrowed with increasing spatial and action-space complexity; short two-arm mazes ([Formula: see text] - 6) supported near-ceiling reward across a broader range, whereas longer arms ([Formula: see text]) required tighter tuning and showed reduced efficiency. In multi-arm mazes, efficient exploration persisted only in the least demanding conditions (e.g., [Formula: see text]), collapsing toward timeouts as arm length and arm number increased. These results link cholinergic-like synaptic depression to flexible exploration while revealing scaling limits in complex environments.

The online version contains supplementary material available at 10.1007/s11571-026-10477-5.