Your Model Is Unfair, Are You Even Aware? Inverse Relationship between Comprehension and Trust in Explainability...

Area of Science:

• Machine Learning
• Human-Computer Interaction
• Responsible AI

Background:

• Machine learning (ML) systems are widespread, yet exhibit biased behavior, impacting user trust and interaction.
• Stakeholder trust and perception of ML systems vary significantly across different backgrounds.
• Explainability visualizations are crucial for understanding ML model behavior, comprehension, and trust.

Purpose of the Study:

• To survey explainability visualizations and create a taxonomy of design characteristics.
• To evaluate state-of-the-art ML explainability visualization tools (LIME, SHAP, CP, Anchors, ELI5) through user studies.
• To measure the impact of visualization design characteristics on comprehension, bias perception, and trust among non-expert ML users.

Main Methods:

• Conducted user studies evaluating five explainability visualization tools.
• Developed a taxonomy of design characteristics for explainability visualizations.
• Measured comprehension, bias perception, and trust in relation to visualization design and model fairness.

Main Results:

• An inverse relationship was found between comprehension and trust: higher comprehension led to lower trust.
• Bias perception mediates this relationship: more comprehensible visualizations increased perceived bias, reducing trust.
• Visualization design significantly impacts comprehension, perceived bias, and trust (p < 0.001).

Conclusions:

• Explainability visualization design can be manipulated to control comprehension, bias perception, and trust.
• Reducing perceived model bias, through fairness improvements or visualization adjustments, increases trust, even with high comprehension.
• Findings advance the understanding of comprehension-trust dynamics and visualization's role in responsible ML.