Interpretable neural networks: principles and applications.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep neural networks (DNNs) achieve high performance but lack transparency.
• The "black-box" nature of DNNs hinders understanding of their decision-making processes.
• Interpretability of neural networks is a critical research area across various domains.

Purpose of the Study:

• To review and categorize current interpretable neural network (INN) methods.
• To explore the application scenarios and future development of INNs.
• To provide a comprehensive overview of the interpretability challenge in deep learning.

Main Methods:

• Categorization of INN methods into "model decomposition" and "semantic INNs".
• Model decomposition INNs integrate conventional analytical models into neural network layers.
• Semantic INNs utilize visualization and semantic information for post-hoc explanation of black-box models.

Main Results:

• Identified two primary directions for interpretable neural networks: model decomposition and semantic INNs.
• Model decomposition INNs are further classified based on derived models (mathematical, physical, etc.).
• Semantic INNs employ visualization techniques like convolutional layer output visualization and decision tree extraction.

Conclusions:

• Interpretable neural networks offer pathways to understand complex deep learning models.
• Both pre-design (model decomposition) and post-hoc (semantic) methods contribute to network interpretability.
• Further research is needed to address existing challenges and advance the field of INNs.