Better Understanding Differences in Attribution Methods via Systematic Evaluations.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Deep neural networks excel in vision tasks but lack interpretability due to their "black box" nature.
• Post-hoc attribution methods aim to identify influential image regions for model decisions, but their evaluation is difficult without ground truth.

Purpose of the Study:

• To develop novel evaluation schemes for reliably measuring the faithfulness and fairness of attribution methods.
• To enable more systematic visual inspection and comparison of different attribution techniques.
• To study the strengths and weaknesses of widely used attribution methods across various models.

Main Methods:

• Introduced DiFull: a controlled evaluation setting to assess attribution faithfulness by manipulating input influence.
• Proposed ML-Att: evaluating all methods on the same network layers to ensure fair comparison.
• Developed AggAtt: a scheme for systematic qualitative evaluation of methods on complete datasets.

Main Results:

• The proposed schemes facilitate a more reliable and fair comparison of attribution methods.
• Analysis revealed strengths and shortcomings of several popular attribution techniques.
• A post-processing smoothing step was found to significantly enhance the performance of certain attribution methods.

Conclusions:

• The novel evaluation schemes provide a robust framework for assessing deep neural network interpretability methods.
• Fairer and more systematic evaluations lead to better understanding and selection of attribution techniques.
• Proposed methods and post-processing steps contribute to advancing the field of explainable AI.