Differentially private knowledge transfer for federated learning

Area of Science:

• Machine Learning
• Data Privacy
• Artificial Intelligence

Background:

• Federated learning enables decentralized data analysis but risks privacy leakage through model parameters.
• Existing federated learning methods struggle to maintain privacy while achieving high performance.

Purpose of the Study:

• To introduce PrivateKT, a novel knowledge transfer method for federated learning.
• To ensure privacy guarantees during knowledge transfer using actively selected public data.
• To enhance the performance of federated learning under strict privacy constraints.

Main Methods:

• Developed PrivateKT, a knowledge transfer framework utilizing small, actively selected public datasets.
• Implemented differential privacy mechanisms to protect sensitive information.
• Evaluated PrivateKT on three diverse datasets.

Main Results:

• PrivateKT significantly reduces the performance gap between centralized and federated learning.
• Achieved up to an 84% reduction in the performance gap under strict differential privacy.
• Demonstrated effective knowledge transfer with robust privacy preservation.

Conclusions:

• PrivateKT offers a promising direction for effective and privacy-preserving knowledge transfer in machine learning.
• The method enables high-quality knowledge extraction from decentralized, privacy-sensitive data.
• Addresses the critical challenge of balancing performance and privacy in federated systems.