Computational Consequences of a Bias toward Short Connections

A fundamental observation in the neurosciences is that the brain is a modular system in which different regions perform different tasks. Recent evidence, however, raises questions about the accuracy of this characterization with respect to neo-nates. One possible interpretation of this evidence is that certain aspects of the modular organization of the adult brain arise developmentally. To explore this hypothesis we wish to characterize the computational principles that underlie the development of modular systems. In previous work we have considered computational schemes that allow a learning system to discover the modular structure that is present in the environment (Jacobs, Jordan, & Barto, 1991). In the current paper we present a complementary approach in which the development of modularity is due to an architectural bias in the learner. In particular, we examine the computational consequences of a simple architectural bias toward short-range connections. We present simulations that show that systems that learn under the influence of such a bias have a number of desirable properties, including a tendency to decompose tasks into subtasks, to decouple the dynamics of recurrent subsystems, and to develop location-sensitive internal representations. Furthermore, the system's units develop local receptive and projective fields, and the system develops characteristics that are typically associated with topographic maps.