A Vision on User-Centered Implementation and Evaluation of Explainable AI for Predicting Hospital-Onset Bacteremia

In recent years, artificial intelligence (AI) has gained momentum in many fields of daily live. In healthcare, AI can be used for diagnosing or predicting illnesses. However, explainable AI (XAI) is needed to ensure that users understand how the algorithm arrives at a decision. In our research project, machine learning methods are used for individual risk prediction of hospital-onset bacteremia (HOB). This paper presents a vision on a step-wise process for implementation and evaluation of user-centered XAI for risk prediction of HOB. An initial requirement analysis revealed first insights on the users' needs of explainability to use and trust such risk prediction applications. The findings were then used to propose step-wise process towards a user-centered evaluation.