Whitening black boxes: Interpretable and explainable DL-based systems for trustworthy healthcare

Artificial intelligence (AI), and specifically deep learning (DL) models, are rapidly gaining traction in healthcare to analyze complex medical images and support clinical decision-making. However, DL models are often considered black boxes due to the lack of a clear explanation when providing predictions. Explainable artificial intelligence (XAI) methods are emerging as an effective way to make models explainable for developers and provide interpretable outputs for clinicians. This review presents a taxonomy of the most widely used XAI methods for image classification, with related benefits and drawbacks. Furthermore, it examines whether the type of classifier affects the choice of an explainability technique and investigates the impact of black boxes on the healthcare environment. The analysis considered papers published between January 2020 and July 2025 in Scopus and Google Scholar, utilizing the PRISMA guidelines to enhance reporting. Sixty-nine papers were identified as suitable for classifying XAI methods in four categories based on backpropagation, perturbation, attention, and concept. The results show increased use of backpropagation-based techniques, which offer simple and intuitive heatmaps. Perturbation-based methods are frequently employed to validate model robustness, but they are computationally expensive. Finally, concept-based and attention-based approaches are less widespread but represent a promising solution towards explanations that align with human semantics and reflect the intrinsic model behavior. Future research should focus on combined approaches and concept methods that generate explanations in the same semantic field as clinicians and are computationally suitable for healthcare environments, paving the way for transparent and clinically reliable DL systems.