SIGN: Statistical Inference Graphs Based on Probabilistic Network Activity Interpretation

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) excel in visual tasks but lack interpretability.
• The opaque nature of intermediate CNN layers hinders trust and understanding.
• Interpreting network decisions is crucial for reliable AI systems.

Purpose of the Study:

• To introduce the SIGN method for modeling hidden layer activity in neural networks.
• To develop interpretable models of network inference processes.
• To enhance trust and understanding of Convolutional Neural Networks (CNNs).

Main Methods:

• Modeling hidden layer activity using probabilistic models, specifically Gaussian mixture models.
• Estimating transition probabilities between clusters in consecutive layers to identify inference paths.
• Developing an explanatory inference graph using a maximum likelihood model for convolutional layers.

Main Results:

• The SIGN method generates inference graphs that reveal the hierarchy of activity clusters relevant for prediction.
• These graphs aid in understanding general class inference and explaining specific image decisions.
• Observations include memorization concentration in fully connected networks and local activity in top CNN layers.

Conclusions:

• The SIGN method provides a powerful tool for interpreting neural network behavior.
• Inference graphs enhance the explainability and trustworthiness of CNNs.
• Further insights into hidden layer activity patterns are revealed by this probabilistic modeling approach.