Guaranteed distributed machine learning: Privacy-preserving empirical risk minimization.

Area of Science:

• Computer Science
• Data Science
• Cybersecurity

Background:

• Distributed learning enables collaborative model training on sensitive data from sensor networks without privacy breaches.
• Existing methods often face challenges in balancing privacy and model accuracy in distributed settings.

Purpose of the Study:

• To present an integrated distributed learning framework combining secure multi-party computation (SMC) and differential privacy (DP).
• To develop and validate advanced DP techniques, specifically output perturbation (OP) and gradient perturbation (GP), for distributed learning.

Main Methods:

• Proposed a Multi-Scheme Output Perturbation (MS-OP) algorithm where data owners securely combine local classifiers and add statistical noise.
• Developed an Adaptive Iterative Gradient Perturbation (MS-GP) method with meticulous privacy budget calibration and a line-search capability for accurate gradient measurement.
• Integrated these methods within a secure multi-party computation domain for collaborative model training.

Main Results:

• The MS-OP algorithm injects noise into combined local models before revealing them.
• The MS-GP algorithm refines gradient measurements during training, especially as model parameters approach optimal values.
• Validation on three real-world datasets demonstrated a competitive advantage over current privacy-preserving distributed learning methods.

Conclusions:

• The proposed framework effectively enhances privacy in distributed learning for sensor-based networks.
• The MS-OP and MS-GP methods offer improved privacy-preserving capabilities compared to existing approaches.
• The framework provides a robust solution for training models on sensitive data while mitigating privacy risks.