Non-negative connectivity causes bow-tie architecture in neural circuits

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Machine Learning

Background:

• Bow-tie architecture (BTA) is prevalent in biological neural networks, but its formation mechanism is not well understood.
• Existing artificial neural network (ANN) models often require pre-defined architectures to achieve BTA.
• Investigating biologically inspired constraints can reveal fundamental principles of neural organization.

Purpose of the Study:

• To identify a novel mechanism for the spontaneous emergence of BTA in multi-layer neural networks.
• To explore the role of biologically inspired non-negative connectivity constraints in network self-organization.
• To demonstrate that non-negativity alone can induce BTA formation.

Main Methods:

• Training multi-layer neural networks with non-negative weight constraints.
• Utilizing diverse classification tasks to test the robustness of the proposed mechanism.
• Analyzing network dynamics, including error signal propagation and hidden-layer activity.
• Comparing the emergent BTA with pre-defined architectures.

Main Results:

• Non-negative weights were shown to amplify back-propagated error signals.
• Suppressed hidden-layer activity was observed under non-negative constraints.
• The self-organization of BTA was achieved without any pre-defined architectural biases.
• This is the first demonstration of BTA formation induced solely by non-negativity.

Conclusions:

• Non-negativity is a sufficient condition for inducing BTA in artificial neural networks.
• The emergent BTA exhibits functional advantages: reduced wiring cost, scalability, and task generalizability.
• Findings bridge principles of artificial learning with the biological relevance of neural structures.
• This study provides a mechanistic explanation for BTA emergence in biological systems.