An Uplink Communication-Efficient Approach to Featurewise Distributed Sparse Optimization With Differential Privacy.

Area of Science:

• Machine Learning
• Data Privacy
• Distributed Systems

Background:

• Training sparse empirical risk minimization (ERM) models with sensitive data requires differential privacy (DP).
• Featurewise distributed multiparty settings, where data is split by features across organizations, are common and scalable but lack existing private sparse optimization methods.
• Current private sparse optimizations are restricted to centralized or samplewise distributed datasets.

Purpose of the Study:

• To develop a differentially private algorithm for sparse ERM model training in featurewise distributed multiparty settings.
• To guarantee differential privacy, achieve near-optimal utility, and reduce uplink communication complexity.
• To extend convergence analysis for block-coordinate Frank-Wolfe (BCFW) algorithms.

Main Methods:

• Developed a novel differentially private algorithm for featurewise distributed sparse ERM.
• Presented a generalized convergence analysis for block-coordinate Frank-Wolfe with arbitrary sampling (BCFW-AS).
• Designed an active private feature sharing scheme using Johnson-Lindenstrauss transformations to minimize communication costs.

Main Results:

• The proposed algorithm guarantees differential privacy and achieves nearly optimal utility.
• The generalized BCFW-AS convergence analysis significantly broadens applicability beyond specific sampling distributions.
• The active private feature sharing scheme effectively reduces uplink communication while ensuring convergence.

Conclusions:

• The new algorithm offers a practical solution for private, distributed sparse model training.
• The extended convergence analysis provides a more robust theoretical foundation for BCFW methods.
• The approach effectively balances privacy, utility, and communication efficiency in challenging distributed environments.