A scoping review of explainable artificial intelligence for medical multimodal data

Area of Science:

• Medical Artificial Intelligence
• Explainable AI (XAI)
• Healthcare Informatics

Background:

• Multimodal AI models integrating diverse data (e.g., imaging, clinical records) are advancing in healthcare.
• A significant gap exists between these complex predictive models and explainable AI (XAI) techniques for interpretation.

Purpose of the Study:

• To conduct a scoping review on the use of explainability methods in cross-modal medical AI studies.
• To identify current trends and gaps in XAI for multimodal medical AI.

Main Methods:

• Scoping review across 4 bibliographic databases.
• Inclusion of 82 studies focusing on explainability in cross-modal medical AI.

Main Results:

• The field is dominated by independent feature attribution and post-hoc XAI methods, treating models as 'black boxes'.
• Emerging trends like visual grounding and model reasoning show promise but a critical gap in explaining the reasoning process remains.
• Standardized evaluation and reproducibility are lacking, with most studies relying on qualitative measures and few providing public codebases.

Conclusions:

• The field needs to move beyond isolated, post-hoc XAI towards intrinsically explainable AI designs.
• Integrating reasoning logic directly into model architecture is crucial for aligning AI outputs with clinical workflows and enhancing trust.