Evaluating the Visualization of What a Deep Neural Network Has Learned.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep neural networks (DNNs) excel at tasks like image classification but lack transparency.
• Interpreting DNN decisions for unseen data is challenging due to their complex structure.
• Existing methods visualize DNN reasoning using heatmaps, but lack objective quality measures.

Purpose of the Study:

• To introduce a general methodology for evaluating pixel-importance heatmaps derived from DNNs.
• To objectively compare different DNN interpretation methods.
• To assess the effectiveness of heatmaps for unsupervised neural network evaluation.

Main Methods:

• Developed a region perturbation methodology to evaluate ordered pixel collections (heatmaps).
• Compared three heatmap computation methods: layer-wise relevance propagation, sensitivity-based approach, and deconvolution.
• Evaluated methods on SUN397, ILSVRC2012, and MIT Places datasets.

Main Results:

• Layer-wise relevance propagation quantitatively and qualitatively outperformed sensitivity-based and deconvolution methods.
• The proposed methodology provides an objective measure for heatmap quality.
• Heatmaps can be utilized for unsupervised assessment of neural network performance.

Conclusions:

• Layer-wise relevance propagation offers a more robust explanation of DNN decisions.
• The region perturbation method provides a valuable tool for evaluating DNN interpretability techniques.
• Further research can leverage these findings for improved DNN transparency and assessment.