Detecting Information Relays in Deep Neural Networks

Area of Science:

• Artificial intelligence
• Computational neuroscience
• Information theory

Background:

• Deep learning models, or artificial neural networks (ANNs), are complex and difficult to interpret.
• Understanding functional modules in biological brains is crucial in neuroscience, but similar methods are lacking for ANNs.
• Interpreting ANNs could help solve problems like catastrophic forgetting and overfitting, increasing trust in AI.

Purpose of the Study:

• To introduce a novel information-theoretic concept, relay information (IR), for analyzing functional modularity in ANNs.
• To develop a method for identifying computational modules within neural networks.
• To enhance the transparency and interpretability of artificial neural networks.

Main Methods:

• Introduced the concept of relay information (IR) to quantify information flow through neuronal groups.
• Combined IR with a greedy search algorithm to identify functional modules in ANNs.
• Correlated the identified modules' functionality with their calculated relay information.

Main Results:

• Relay information (IR) was shown to be a useful metric for analyzing functional modularity in ANNs.
• A greedy search algorithm combined with IR successfully identified computational modules.
• The functionality of identified modules was found to correlate with the amount of relay information they processed.

Conclusions:

• Relay information provides a novel way to understand and analyze the functional organization of artificial neural networks.
• This approach can aid in identifying computational modules, thereby increasing ANN transparency and interpretability.
• The findings suggest that IR can be a valuable tool for addressing key challenges in artificial intelligence research.