An attribution graph-based interpretable method for CNNs.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) excel in image-related tasks but lack transparency, hindering applications in critical domains like medical diagnosis and autonomous driving.
• Existing interpretability methods for CNNs primarily focus on feature-based analysis, with limited attention to the network's overall structure and parameter roles.
• Understanding the internal workings of CNNs is crucial for building trust and reliability in high-stakes decision-making processes.

Purpose of the Study:

• To propose a novel method, Attribution Graph-based Interpretable method for CNNs (AGIC), for enhancing the interpretability of Convolutional Neural Networks.
• To model the overall structure of CNNs as attribution graphs (At-GCs) for both global and local interpretability.
• To intuitively comprehend the role of internal parameters and convolutional kernels within CNNs.

Main Methods:

• Constructed attribution graphs (At-GCs) using runtime parameters and feature maps, representing convolutional kernels as nodes and SHAP values as edges.
• Pretrained a heterogeneous graph encoder using Deep Graph Infomax (DGI) on the generated At-GCs.
• Utilized the pretrained encoder for two tasks: classifying At-GCs to reveal category-dependent topological characteristics and employing a scoring aggregation (SA) network to assess kernel importance.

Main Results:

• Attribution graph topological characteristics demonstrated a dependency on image sample categories, indicating distinct kernel activation patterns for different categories.
• The scoring aggregation network successfully identified crucial kernels for feature extraction and highlighted kernels that could be pruned without performance degradation.
• The AGIC method provides a more comprehensive understanding of CNN structure and parameter significance.

Conclusions:

• The proposed AGIC method effectively enhances the interpretability of CNNs by analyzing their graph-based structure and kernel importance.
• The findings suggest that CNNs exhibit category-specific structural patterns, offering insights into their decision-making processes.
• AGIC facilitates the identification of essential kernels, paving the way for more efficient and interpretable CNN models in critical applications.