From explainable to interpretable deep learning for natural language processing in healthcare: How far from reality?

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Healthcare Informatics

Background:

• Deep learning (DL) significantly advances healthcare NLP but requires transparency for reliable decision-making.
• Increasing DL model complexity necessitates explainability and interpretability.
• This scoping review focuses on explainable and interpretable DL within healthcare NLP.

Purpose of the Study:

• To conduct a comprehensive scoping review of explainable and interpretable DL in healthcare NLP.
• To introduce and define the term 'eXplainable and Interpretable Artificial Intelligence' (XIAI).
• To categorize models by functionality and scope, identifying current trends and challenges.

Main Methods:

• Systematic literature search for explainable and interpretable DL in healthcare NLP.
• Categorization of models based on functionality (model-, input-, output-based) and scope (local, global).
• Analysis of identified techniques, trends, challenges, and opportunities.

Main Results:

• Attention mechanisms are the most prevalent emerging interpretable AI (IAI) technique.
• IAI adoption is growing, distinct from explainable AI (XAI).
• Key challenges include limited exploration of global modeling, lack of best practices, and insufficient systematic evaluation.

Conclusions:

• Integrating XIAI into Large Language Models (LLMs) and smaller domain-specific models is encouraged.
• Adoption of XIAI in healthcare requires in-house expertise and collaboration with stakeholders.
• Despite challenges, XIAI offers a valuable foundation for interpretable NLP in healthcare.