Geometry and efficiency of learned and reservoir recurrent dynamics in context-dependent integration-switching

The computational strategy of recurrent neural networks (RNNs) is encoded in the geometry of their state-space dynamics. We investigate the fundamental differences between two dominant paradigms: fixed-reservoir computing and end-to-end trainable RNNs. On a canonical context-dependent integration task, we systematically compare echo-state networks, gated recurrent units, long short-term memory networks, and simplified linear models. We find that trainable RNNs consistently achieve superior accuracy with significantly fewer parameters. Using methods from dynamical systems and manifold analysis, we uncover the mechanism for this efficiency: trainable networks learn to sculpt their internal dynamics, creating low-dimensional, geometrically organized manifolds that are aligned with the task's computational requirements. In contrast, fixed reservoirs rely on a high-dimensional, entangled representation that is less efficient. These findings, supported by intrinsic dimensionality and spectral analysis, demonstrate that learning to distill task structure into a compact, minimal realization is a hallmark of efficient recurrent computation and connects theoretical principles of universality with biological observations of neural remapping.