Designing effective explainable AI: a human-centered evaluation of explanation formats in financial decision-making

Area of Science:

• Artificial Intelligence
• Human-Computer Interaction
• Financial Technology

Background:

• Artificial intelligence (AI) systems are increasingly used in high-risk financial decision-making.
• Explainable AI (XAI) is crucial for transparency and interpretability in these applications.
• Existing XAI research often prioritizes technical metrics over end-user needs and diverse stakeholder perspectives.

Purpose of the Study:

• To conduct a human-centered evaluation of visual explanation designs for financial AI.
• To assess the effectiveness of different XAI visualization types based on user and stakeholder preferences.
• To identify trade-offs between interpretability and completeness in AI explanations.

Main Methods:

• A two-phase mixed-method evaluation was employed.
• Phase one involved user studies with end-users of financial AI applications.
• Phase two included a stakeholder workshop with compliance officers, XAI consultants, and developers.

Main Results:

• A significant divergence in preferences was observed between end-users and other stakeholders.
• End-users favored concise, contextually visual explanations like decision rules or risk plots.
• Other stakeholders preferred technically detailed and complete representations of AI explanations.

Conclusions:

• Visual encoding choices in XAI significantly impact effectiveness across different stakeholder groups.
• A critical trade-off exists between interpretability and completeness in AI explanations.
• Future XAI design must consider diverse stakeholder requirements for effective deployment in finance.