Unexpected benefits of self-modelling in neural systems

Self-models have been a topic of interest for decades in human cognition and more recently in machine learning. Yet what benefits do self-models confer? Here we show that when an artificial network learns to predict its internal states as an auxiliary task, the network becomes simpler, more regularized and more parameter efficient. To test the hypothesis of self-regularizing through self-modelling, we used a range of network architectures performing three classification tasks across two modalities. In all cases, adding self-modelling caused a significant reduction in network complexity. The reduction was observed in two ways. First, the distribution of weights was narrower when self-modelling was present. Second, a measure of network complexity, the real log canonical threshold (RLCT), was smaller when self-modelling was present. These results support the hypothesis that self-modelling has a restructuring effect, reducing complexity and increasing parameter efficiency. This self-regularization may help explain some of the benefits of self-models reported in recent machine learning literature, as well as the adaptive value of self-models to biological systems. In particular, these findings may shed light on the possible interaction between the ability to model oneself and the ability to be more easily modelled by others in a social or cooperative context. This article is part of the theme issue 'World models in natural and artificial intelligence'.