Distill to Delete: Unlearning in Graph Networks With Knowledge Distillation

Area of Science:

• Artificial Intelligence
• Machine Learning
• Graph Neural Networks

Background:

• Graph unlearning enables information deletion from trained graph neural networks (GNNs), crucial for data privacy and model adaptability.
• Existing methods struggle with complex graph dependencies and incur significant overhead.
• The need for efficient and effective graph unlearning is driven by privacy regulations and dynamic data environments.

Purpose of the Study:

• To introduce a novel, efficient, and model-agnostic graph unlearning framework called D2DGN.
• To address limitations of existing methods in handling local graph dependencies and overhead costs.
• To effectively delete specific graph elements while preserving knowledge of retained elements.

Main Methods:

• Developed D2DGN, a knowledge distillation framework for graph unlearning.
• Implemented a strategy to divide graph knowledge into retention and deletion sets.
• Utilized response-based soft targets and feature-based node embeddings with KL-divergence minimization.

Main Results:

• D2DGN demonstrated superior performance in node and edge unlearning tasks, outperforming existing methods by up to 43.1% (AUC).
• Achieved high efficiency, improved removal of target elements, and preserved performance on retained data.
• Showcased zero overhead costs, making it a highly efficient solution.

Conclusions:

• D2DGN offers an effective and efficient solution for graph unlearning in GNNs.
• The knowledge distillation approach successfully balances information deletion and retention.
• D2DGN provides a practical framework for complying with data protection regulations and managing evolving graph data.