A mental models approach for defining explainable artificial intelligence.

Area of Science:

• Artificial Intelligence
• Human-Computer Interaction
• Healthcare Informatics

Background:

• Black-box AI models face significant adoption barriers in sensitive fields like healthcare due to trust and transparency concerns.
• Existing definitions of explainable AI lack the specificity needed for practical implementation and evaluation.
• The hype surrounding AI performance gains is tempered by the need for understandable and trustworthy systems.

Purpose of the Study:

• To critique existing literature on AI agency, mental models in healthcare, and definitions of explainability.
• To propose a novel, context-dependent definition of explainability for AI systems.
• To provide a framework for objectively evaluating explainability in AI models.

Main Methods:

• Literature review focusing on AI agency, mental models, and explainability definitions.
• Development of a new definition of explainability based on context (purpose, audience, language).
• Application of the new definition to regression models, neural networks, and human mental models.

Main Results:

• Current explainability definitions fail to address the "understandable by whom?" question.
• Explainability can be defined by the model's context: its purpose, intended audience, and language.
• The proposed definition was successfully applied to diverse AI models and human team interactions.

Conclusions:

• Existing definitions of explanation are inadequate for resolving practical application concerns.
• Context-specific definitions of explainability align evaluations with practical goals.
• This approach facilitates clear distinctions between explanations for technical and lay audiences.