Consensus-Driven Distillation for Trustworthy Explanations in Self-Interpretable GNNs

Area of Science:

• Artificial Intelligence
• Machine Learning
• Graph Neural Networks

Background:

• Self-interpretable graph neural networks (SI-GNNs) offer inherent explanations but suffer from inconsistency across models.
• Redundant, uninformative features are a key cause of this inconsistency.
• Explanation Ensemble (EE) improves consistency and accuracy by averaging explanations from multiple models.

Purpose of the Study:

• To introduce Consensus-driven Distillation (CD), a novel framework to improve SI-GNN explanation consistency and accuracy.
• To overcome the computational limitations and single-model metric incompatibility of EE.
• To provide a general and theoretically grounded method for distilling ensemble knowledge into a single SI-GNN.

Main Methods:

• Developed Consensus-driven Distillation (CD) to use ensemble-derived consensus explanations as supervision signals.
• Trained a single SI-GNN to assign importance scores more responsibly using CD.
• Evaluated CD across diverse benchmark datasets and SI-GNN frameworks.

Main Results:

• CD effectively retains the benefits of EE while overcoming its limitations.
• Experiments demonstrate CD's ability to improve explanation consistency and accuracy.
• Combining CD with EE further enhances performance gains.

Conclusions:

• CD offers a computationally efficient and effective solution for improving SI-GNN explanations.
• The proposed method is general, theoretically grounded, and compatible with standard metrics.
• CD represents a significant advancement in creating reliable and interpretable graph neural networks.