Explaining Neural Networks Using Attentive Knowledge Distillation.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Deep neural networks (DNNs) are increasingly used in critical applications, but their complexity hinders interpretability.
• Current methods for explaining DNN predictions, like saliency maps, often rely on limited information from the final network layer, leading to coarse explanations.
• Iterative refinement of saliency maps improves accuracy but is computationally expensive and impractical for real-time applications.

Purpose of the Study:

• To develop a novel, efficient, and accurate method for explaining DNN predictions.
• To generate fine-grained saliency maps by leveraging information from all network layers.
• To improve the trustworthiness and applicability of DNNs in mission-critical domains.

Main Methods:

• Proposed an attentive surrogate network trained using knowledge distillation.
• Integrated information from intermediate network layers to create a more comprehensive understanding of model predictions.
• Developed a method to generate fine-grained saliency maps that capture spatially attentive features.

Main Results:

• The proposed method generates fine-grained saliency maps that are more accurate than existing approaches.
• The saliency maps effectively highlight spatially attentive features learned during knowledge distillation.
• The method achieves a processing speed of 24.3 frames per second, significantly outperforming existing techniques.

Conclusions:

• The attentive surrogate network effectively explains DNN predictions by utilizing multi-layer information.
• The generated saliency maps are valuable for fine-grained classification tasks.
• The proposed approach offers a practical and efficient solution for enhancing DNN interpretability.