Federated Learning With Privacy-Preserving Ensemble Attention Distillation

Area of Science:

• Artificial Intelligence
• Machine Learning
• Medical Informatics

Background:

• Federated Learning (FL) enables collaborative model training on decentralized data, crucial for clinical applications due to data privacy regulations.
• Existing FL methods face challenges with data imbalance, require extensive communication, and pose privacy leakage risks.
• The need for robust privacy preservation in FL for sensitive medical data is paramount.

Purpose of the Study:

• To propose a novel privacy-preserving Federated Learning (FL) framework.
• To reduce privacy leakage risks inherent in traditional FL methods.
• To leverage unlabeled public data for effective knowledge distillation in FL.

Main Methods:

• Developed a privacy-preserving FL framework utilizing one-way offline knowledge distillation.
• Employed ensemble attention distillation to learn the central model from local knowledge.
• Utilized decentralized and heterogeneous local data, similar to existing FL approaches.

Main Results:

• Significantly reduced the risk of privacy leakage compared to existing FL methods.
• Achieved competitive performance across image classification, segmentation, and reconstruction tasks.
• Demonstrated robust privacy preservation through extensive experimental validation.

Conclusions:

• The proposed privacy-preserving FL framework offers a viable solution for clinical applications.
• Offline knowledge distillation with ensemble attention is effective for privacy-preserving FL.
• The method provides a strong balance between performance and enhanced privacy preservation.