Scrutinizing XAI using linear ground-truth data with suppressor variables.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Explainable AI (XAI)

Background:

• Complex machine learning models are often "black boxes", necessitating explainable AI (XAI) techniques.
• Saliency methods in XAI rank input features by importance, but lack formal validation and can highlight irrelevant "suppressor variables".

Purpose of the Study:

• To propose an objective, preliminary definition for feature importance based on statistical association with the prediction target.
• To evaluate the performance of common XAI saliency methods against this new definition, particularly concerning suppressor variables.

Main Methods:

• Developed a ground-truth dataset with well-defined, linear statistical dependencies.
• Evaluated multiple XAI methods (LRP, DTD, PatternNet, PatternAttribution, LIME, Anchors, SHAP, permutation-based) on the benchmark dataset.
• Assessed the ability of these methods to differentiate statistically important features from suppressor variables.

Main Results:

• Most evaluated XAI saliency methods failed to distinguish between statistically important features and suppressor variables.
• The proposed objective definition highlighted limitations in current feature importance assessment within XAI.

Conclusions:

• A formal definition of feature importance is crucial for reliable XAI.
• Current popular XAI saliency methods require refinement to accurately identify true feature importance and avoid misinterpretations from suppressor variables.