Decoupling forgetting and preservation in federated unlearning via knowledge distillation

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Privacy

Background:

• Federated Learning (FL) trains models collaboratively without raw data sharing.
• Data privacy regulations necessitate removing specific client data post-training (unlearning).
• Selective unlearning faces challenges in removing one client's influence while preserving others' contributions and overall model utility.

Purpose of the Study:

• To develop a selective federated unlearning framework addressing the conflict between forgetting and preservation.
• To ensure compliance with privacy regulations like the 'Right to be Forgotten' in FL.

Main Methods:

• Proposed a Three-Phase Selective Unlearning framework: aggressive forgetting, friend restoration via knowledge distillation, and balanced fine-tuning.
• Decoupled forgetting and preservation into sequential phases to avoid gradient conflicts.
• Conducted experiments on diverse datasets (MNIST, CIFAR-10/100, Tiny-ImageNet) and architectures (CNNs, ResNet-18) under non-IID settings.

Main Results:

• Achieved target metrics (≥70% forget accuracy drop, ≥70% friend accuracy) on MNIST.
• Demonstrated scalability on Tiny-ImageNet with 83.6% forget accuracy drop and 76.43% friend accuracy.
• Membership inference attacks confirmed effective forgetting, validating the approach.

Conclusions:

• Phase decomposition combined with knowledge distillation is crucial for effective selective federated unlearning.
• The proposed framework successfully balances data removal with performance preservation in FL.