Signal or noise? Evaluating commonly used attribution methods for explaining deep neural networks in...

Area of Science:

• Artificial Intelligence in Healthcare
• Medical Imaging and Diagnostics
• Computational Cardiology

Background:

• Deep neural networks (DNNs) are increasingly used for electrocardiogram (ECG) classification.
• Attribution methods are crucial for interpreting DNN predictions in clinical settings.
• Evaluating the reliability of these explainability techniques is essential for safe deployment.

Purpose of the Study:

• To assess the clinical utility of 12 attribution methods for DNN-based ECG classification.
• To evaluate the clarity and faithfulness of explanations generated by these methods.
• To determine the reliability and stability of attribution methods in medical AI.

Main Methods:

• Analysis of 12 attribution methods applied to convolutional neural network models.
• Utilized a large dataset of 873,710 median beat ECGs across nine diagnostic classes.
• Performance evaluated via inter-method similarity, self-consistency, weight dependence, and feature identification.

Main Results:

• Deep learning models achieved high diagnostic performance (AUC > 0.95).
• Attribution methods exhibited low inter-method correlation and high variability.
• Moderate self-consistency (mean correlation 0.41-0.65) and instability with model weight changes were observed.

Conclusions:

• Attribution methods show limited reliability and instability, constraining their use in critical healthcare applications.
• Task-specific validation and inherently interpretable models are recommended for clinical decision support.
• Cautious application of current attribution techniques, supplemented by sanity checks, is advised.