What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods.

Area of Science:

• Artificial Intelligence
• Human-Computer Interaction
• Cognitive Science

Background:

• Numerous AI explainability methods exist to demystify AI decision-making.
• Existing performance metrics for explainability lack end-user focus and real-world validation.
• The practical utility of explainability techniques and the accuracy of their evaluation metrics remain unclear.

Purpose of the Study:

• To assess the real-world usefulness of current AI explainability methods.
• To determine if existing performance metrics accurately gauge end-user utility.
• To investigate the efficacy of attribution methods across diverse applications.

Main Methods:

• Conducted large-scale psychophysics experiments with 1,150 participants.
• Evaluated representative attribution methods in three distinct real-world scenarios.
• Collected data on user understanding and interaction with AI systems.

Main Results:

• The usefulness of individual attribution methods significantly varied across the evaluated scenarios.
• Human participants' ability to understand AI systems differed based on the explanation method used.
• No single attribution method proved universally effective for enhancing user comprehension.

Conclusions:

• Current explainability methods offer variable utility in practical settings.
• Existing metrics may not adequately capture the end-user value of AI explanations.
• Future research should focus on developing complementary, qualitatively diverse explanation strategies.