Have We Learned to Explain?: How Interpretability Methods Can Learn to Encode Predictions in their Interpretations.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Explainable AI

Background:

• Interpretable machine learning is crucial but existing methods face challenges with speed, fidelity, and evaluation.
• Amortized explanation methods offer a solution by learning a global selector model for efficient instance-specific feature importances.
• Current popular methods may inadvertently encode predictions within interpretations, compromising true explainability.

Purpose of the Study:

• To introduce EVAL-X, a novel method for quantitatively evaluating the fidelity of machine learning interpretations.
• To present REAL-X, an advanced amortized explanation method designed for improved accuracy and efficiency.
• To address the issue of predictions being encoded within interpretations by popular existing methods.

Main Methods:

• Developed EVAL-X to quantitatively assess interpretation fidelity by detecting encoded predictions.
• Introduced REAL-X, an amortized explanation technique learning a predictor model approximating the true data distribution.
• Trained REAL-X to approximate the data generating distribution given any input subset for enhanced interpretability.

Main Results:

• Demonstrated EVAL-X's capability to detect when predictions are encoded within machine learning interpretations.
• Showcased the advantages of REAL-X through quantitative evaluations and assessments by radiologists.
• REAL-X exhibited superior performance compared to existing amortized explanation methods.

Conclusions:

• EVAL-X provides a critical tool for validating the trustworthiness of machine learning explanations.
• REAL-X represents a significant advancement in amortized explanation methods, offering improved fidelity and efficiency.
• The developed methods enhance the reliability and practical applicability of interpretable machine learning in real-world scenarios, including medical imaging analysis.