Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Explainable AI (XAI) methods aim to demystify complex machine learning models.
• Current explanation techniques, like heatmaps, often entangle multiple decision factors.
• There is a need to disentangle these factors for clearer model understanding.

Purpose of the Study:

• To develop novel methods for disentangling explanations from neural network intermediate layers.
• To identify and isolate distinct activation patterns relevant to model predictions.
• To improve the interpretability of complex machine learning models.

Main Methods:

• Proposed Principal Relevant Component Analysis (PRCA) and Disentangled Relevant Subspace Analysis (DRSA).
• Extended principles from Principal Component Analysis (PCA) and Independent Component Analysis (ICA) to explanation analysis.
• Focused on maximizing relevance of extracted subspaces to model predictions, rather than variance or kurtosis.

Main Results:

• PRCA and DRSA effectively extract subspaces capturing distinct, relevant activation patterns.
• The methods successfully disentangle multiple factors contributing to a model's decision.
• Demonstrated practical utility and favorable comparison to state-of-the-art on benchmarks and use cases.

Conclusions:

• The proposed PRCA and DRSA offer a powerful approach to disentangle AI explanations.
• These methods enhance focus on model-relevant features, ignoring invariant activations.
• The approach is compatible with existing attribution techniques like Shapley Value, Integrated Gradients, and LRP.