Trading off accuracy and explainability in AI decision-making: findings from 2 citizens' juries

Area of Science:

• Artificial Intelligence (AI)
• Public Policy
• Human-Computer Interaction

Background:

• Public perception of AI systems, particularly the trade-offs between accuracy and explainability, is critical for responsible development.
• Existing assumptions about public preferences for AI explainability may not align with real-world values, especially in sensitive domains like healthcare.

Purpose of the Study:

• To investigate public trade-offs between AI explainability and accuracy.
• To determine if these trade-offs differ between healthcare and non-healthcare applications.

Main Methods:

• Utilized citizens' juries, a deliberative democracy method, with 18 jurors per jury in the UK.
• Jurors evaluated three AI systems with varying accuracy and explainability levels across four scenarios (two healthcare, two non-healthcare).
• Quantitative voting and qualitative thematic analysis of juror deliberations and comments were employed.

Main Results:

• In healthcare scenarios, participants prioritized AI accuracy over explainability.
• In non-healthcare scenarios, explainability was valued equally to or more than accuracy.
• Reasons for prioritizing accuracy included societal impact and service efficiency; reasons for valuing explainability included learning, improvement, and bias detection.

Conclusions:

• Public valuation of AI explainability in healthcare may be lower than often assumed by professionals, especially when balanced against accuracy.
• Active public consultation is recommended for developing policies on AI explainability in various domains.