Examining the Causal Structures of Deep Neural Networks Using Information Theory

Area of Science:

• Artificial Intelligence
• Machine Learning
• Information Theory

Background:

• Deep Neural Networks (DNNs) are typically analyzed by their input-output responses.
• Analyzing the internal causal structure of DNNs, or 'what does what', is less explored.
• DNN generalizability is fundamentally linked to its causal structure and response to novel inputs.

Purpose of the Study:

• To introduce information-theory-based metrics for quantifying and tracking DNN causal structure during training.
• To assess the causal influence of nodes and edges within DNN layers.
• To provide tools for enhancing DNN generalizability and explainability.

Main Methods:

• Introduced 'effective information' (EI) as the mutual information between layer input and output after a maximum-entropy perturbation.
• Decomposed EI to measure layer sensitivity (perturbation transmission) and degeneracy (edge overlap interference).
• Estimated integrated information within layers to define a 'causal plane' for visualization.

Main Results:

• Developed metrics to quantify causal influence, sensitivity, and degeneracy within DNN layers.
• Demonstrated visualization of how layer connectivity evolves during training on the 'causal plane'.
• Showed changes in integration and differentiation of layer-by-layer causal structure over time.

Conclusions:

• The proposed metrics offer a novel approach to understanding DNN internal causal mechanisms.
• Analyzing causal structure provides insights into DNN generalization capabilities.
• These foundational tools can lead to more generalizable and explainable DNNs.