A comparative study of nonlinear cleanup rules in resonator networks

Hopfield networks retrieve stored patterns through recurrent nonlinear updates, and later associative-memory variants broaden this design space through alternative nonlinear retrieval rules. Resonator networks use an analogous iterative procedure to solve vector-symbolic factorization: each factor estimate is updated by unbinding the current estimates of the other factors and applying a cleanup rule against the corresponding codebook. This perspective suggests a family of resonator networks indexed by the cleanup nonlinearity. We study four variants-sign cleanup, ReLU cleanup, polynomial cleanup, and softmax cleanup-and compare their factorization capacity, terminal failure modes, convergence behavior, empirical complexity, hyperparameter sensitivity, matched-dimensionality FHRR behavior, sign-projection choices, internal update noise, and neural-output decomposition on visual scene data. Across these evaluations, the cleanup nonlinearity changes both the capacity transition and the dominant failure mode. These results motivate treating resonator networks as a family of modern Hopfield-inspired factorization dynamics rather than as a single update rule.