Toward Faithful Neural Network Intrinsic Interpretation With Shapley Additive Self-Attribution.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Explainable AI (XAI)

Background:

• Current self-interpreting neural networks often lack theoretical grounding for interpretability and limit model expressiveness.
• Existing methods struggle to provide genuine, theoretically sound explanations for model predictions.

Purpose of the Study:

• To propose a generic additive self-attribution (ASA) framework to unify existing approaches.
• To introduce a novel Shapley Additive Self-Attributing Neural Network (SASANet) that incorporates Shapley value attribution for enhanced interpretability.
• To address the limitations of current self-interpreting models by improving both interpretability and performance.

Main Methods:

• Developed a novel Shapley Additive Self-Attributing Neural Network (SASANet).
• Designed an intermediate sequential schema utilizing marginal contributions (MCs) and an internal distillation procedure.
• Theoretically proved that the intermediate self-attribution values converge to the output's Shapley values.

Main Results:

• SASANet achieves high interpretability and outperforms existing self-attributing models in performance.
• SASANet's performance is comparable to commonly used closed-box models.
• The self-attribution method in SASANet offers more accurate and efficient interpretations than post hoc methods.

Conclusions:

• SASANet is the first self-interpreting neural network structure to achieve model-wise Shapley attribution.
• The proposed framework enhances both the interpretability and performance of neural networks.
• SASANet offers a theoretically sound and practically effective approach to explainable AI.