Seeing Is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Current neural networks often lack interpretability, hindering understanding of their decision-making processes.
• Biological systems exhibit modularity and efficient connectivity, offering inspiration for AI design.
• Existing methods for neural network training do not sufficiently prioritize modularity and interpretability.

Purpose of the Study:

• To introduce Brain-Inspired Modular Training (BIMT), a novel method for enhancing neural network modularity and interpretability.
• To explore the potential of geometric embedding and connection cost minimization for AI model understanding.
• To demonstrate BIMT's effectiveness across various datasets and tasks.

Main Methods:

• Developed BIMT, which embeds neurons in a geometric space and adds a connection length cost to the loss function.
• Inspired by evolutionary biology's minimum connection cost principle, adapted for gradient descent training.
• Applied Newman's modularity measure to quantitatively assess network structure.

Main Results:

• BIMT successfully discovers modular neural networks for diverse tasks, revealing compositional structures and interpretable features.
• Qualitative analysis shows BIMT-trained networks possess visually identifiable modules, unlike standard networks.
• Quantitative evaluation using Newman's method confirms BIMT achieves superior modularity across all tested problems.

Conclusions:

• BIMT offers a principled approach to creating inherently modular and interpretable neural networks.
• The method demonstrates significant potential for understanding complex AI systems.
• Future work includes applying BIMT to large-scale models in vision, language, and scientific domains.