Working memory and the need for explainable AI - Scenarios from healthcare, social media and insurance

Explainable AI (XAI) is discussed as an important feature of AI systems that is required for professional, ethical and commercial reasons. XAI is particularly needed in AI systems that remain relatively closed, i.e. black box, on how to make decisions whether for complexity or commercial sensitivity. Still, many factors play an important role on whether people perceive a need for XAI. The personal relevance of the AI system might play a role, but also individual differences in personality traits or cognitive functions. In order to test these assumptions, 215 participants (129 female) underwent an online experiment testing their objective working memory capacity and completed the Intellect facet of the Big Five Personality traits (as a proxy for self-assessed cognitive capacity). Further, all participants were presented three AI systems scenarios of relatively different degrees of (personal) relevance and, hence, personal investment: skin cancer detection (i), the personalization of social media feeds (ii), and the amount to pay for a car insurance (iii). We observed that people reported a higher need for XAI in scenarios where they were more personally invested (health, payment for insurance > social media); for signficant differences between these scenarios see the result section. Interestingly, working memory capacity was not associated with XAI needs across the three domains, but self-reported Intellect showed a mild positive association with need for an AI explanation in the context of the insurance story. The present study represents a first attempt to investigate objective working memory capacity in the context of need for XAI. Beyond that, the study indicates that personal investment is crucial for determining the need for XAI.