Classification Under Local Differential Privacy with Model Reversal and Model Averaging

Area of Science:

• Computer Science
• Data Privacy
• Machine Learning

Background:

• Local differential privacy (LDP) offers robust data protection at the source.
• LDP's inherent noise often degrades data utility, posing challenges for analysis.
• Existing methods lack effective strategies to balance LDP privacy with data utility.

Purpose of the Study:

• To improve classification performance in the context of Local Differential Privacy (LDP).
• To address the utility-privacy trade-off inherent in LDP mechanisms.
• To propose novel techniques for private learning under LDP.

Main Methods:

• Reinterpreting private learning under LDP as a transfer learning problem (noisy data as source, clean data as target).
• Developing a noised binary feedback mechanism for estimating dataset utility.
• Implementing model reversal to salvage underperforming classifiers.
• Utilizing model averaging with utility-based weights for enhanced performance.

Main Results:

• Theoretical excess risk bounds under LDP were established and demonstrated to be reduced by the proposed methods.
• Empirical validation on simulated and real-world datasets confirmed substantial improvements in classification accuracy.
• The novel techniques effectively mitigate the utility loss associated with LDP.

Conclusions:

• The proposed transfer learning approach significantly enhances classification accuracy under LDP.
• Model reversal and averaging are effective strategies for improving utility in LDP settings.
• This work provides a practical framework for achieving high utility while maintaining strong privacy guarantees.