Differential Privacy Enabled Robust Asynchronous Federated Multitask Learning: A Multigradient Descent Approach.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated learning (FL) offers privacy-preserving deep learning but struggles with data/device heterogeneity, information leakage, and communication constraints.
• Existing FL frameworks face practical limitations due to issues like non-convex loss functions and the need for robust aggregation methods.

Purpose of the Study:

• To introduce a federated multitask learning (FedMTL) approach to address FL challenges.
• To develop a semi-asynchronous model aggregation method for improved efficiency and robustness.
• To enhance privacy protection in FL using distributed differential privacy with convergence guarantees.

Main Methods:

• Reformulated FL as a multiobjective optimization problem, leading to the federated multigradient descent algorithm (FedMGDA).
• Developed a semi-asynchronous aggregation method to mitigate straggler and staleness effects.
• Applied distributed differential privacy to asynchronous FedMGDA, analyzing convergence for convex and non-convex loss functions (DP-AsynFedMGDA).

Main Results:

• The proposed FedMGDA enhances model personalization against data heterogeneity and Byzantine attacks.
• The semi-asynchronous aggregation method effectively compensates for straggler and staleness impacts.
• DP-AsynFedMGDA demonstrates improved privacy protection with proven convergence guarantees for various loss functions.

Conclusions:

• The DP-AsynFedMGDA approach effectively addresses key challenges in federated learning, including privacy, heterogeneity, and efficiency.
• The study validates the effectiveness of the proposed methods through empirical examples and comparative analyses.
• This work contributes a more practical and robust federated learning framework suitable for real-world applications.