On redundancy in neural architecture: dynamics of a simple module-based neural network and initial-state independence

This article discusses the relationship between redundancy in neural architecture and activity (cell output or internal state) dynamics with a simple module-based neural network. In the network, a single neural cell with self-feedback is employed as a module sub-network, and all module sub-networks are connected via inter-module connections. In general, the activity dynamics of a single neural cell with positive self-feedback may have two minima in its energy surface, and the minimum the cell state converges to depends on the initial states. However, in the module-based network with all the same intra-module connections, an independence from initial states becomes conspicuous as the number of modules increases due to the architectural redundancy. Simulation and analytical studies on the network dynamics illustrate that the cell states always converge to a global minimum irrelevantly of the initial cell-states, and they never go to a local minimum when a sufficient number of modules are employed.