Mitigating belief projection in explainable artificial intelligence via Bayesian teaching.

Area of Science:

• Artificial Intelligence
• Cognitive Science
• Human-Computer Interaction

Background:

• Deep learning models exhibit complex decision-making processes difficult for humans to interpret.
• Existing Explainable AI (XAI) methods often overlook human cognitive biases and reasoning patterns when explaining AI behavior.

Purpose of the Study:

• To develop and evaluate a novel approach, Bayesian teaching, for generating AI explanations that align with human reasoning.
• To enhance human users' ability to understand and predict the judgments of AI systems.

Main Methods:

• Bayesian teaching was implemented to model the human 'explainee' and evaluate explanations based on their ability to shift user inferences towards a target goal.
• The approach was tested in a binary image classification task, comparing user predictions with and without Bayesian teaching interventions.
• Explanations were generated using both whole examples and sub-examples (saliency maps) to assess their complementary roles.

Main Results:

• Bayesian teaching effectively shifted participants' prior beliefs about AI classifications, improving their accuracy in predicting AI judgments.
• Sub-examples (saliency maps) enhanced error detection for familiar categories, while whole examples aided in predicting AI performance on unfamiliar cases.
• Explanations generated via Bayesian teaching demonstrated a significant improvement in user understanding compared to baseline conditions.

Conclusions:

• Explicitly modeling the human explainee through Bayesian teaching offers a more effective strategy for improving AI explainability.
• A combination of whole and sub-examples provides a comprehensive approach to enhancing human understanding of AI decision-making.
• This framework advances the development of more intuitive and effective human-AI collaboration tools.