Differentially Private Empirical Risk Minimization.

Area of Science:

• Machine Learning
• Data Privacy
• Computer Science

Background:

• Analyzing personal data (medical, financial) necessitates robust privacy-preserving machine learning algorithms.
• Empirical risk minimization (ERM) is a common framework for learning classifiers.
• Existing methods like output perturbation offer limited privacy-performance trade-offs.

Purpose of the Study:

• To develop novel, privacy-preserving machine learning algorithms for analyzing sensitive datasets.
• To introduce and evaluate the "objective perturbation" method for enhanced data protection.
• To provide end-to-end privacy guarantees throughout the machine learning model training process.

Main Methods:

• Applied output perturbation techniques from Dwork et al. (2006) to ERM classification.
• Proposed and detailed the novel "objective perturbation" method, altering the objective function before optimization.
• Developed privacy-preserving parameter tuning techniques for comprehensive training process privacy.
• Proved theoretical privacy guarantees under ε-differential privacy and derived generalization bounds for linear/nonlinear kernels.

Main Results:

• Demonstrated that objective perturbation is theoretically and empirically superior to output perturbation.
• Achieved better management of the privacy-performance trade-off.
• Successfully applied the methods to create privacy-preserving logistic regression and support vector machines.
• Obtained encouraging performance results on real demographic and benchmark datasets.

Conclusions:

• Objective perturbation represents a significant advancement in privacy-preserving machine learning.
• The proposed techniques offer strong privacy guarantees while maintaining competitive learning performance.
• This work provides practical, end-to-end privacy solutions for sensitive data analysis.