Differentially private data augmentation via LLM generation with discriminative and distribution-aligned filtering

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Data augmentation (DA) is crucial for mitigating data insufficiency but poses privacy risks in sensitive domains.
• Existing privacy-preserving text generation methods lack formal guarantees.
• Differential Privacy (DP) offers theoretical guarantees but often degrades synthesis quality in text generation.

Purpose of the Study:

• To develop a novel DP-based data augmentation framework for private-domain text generation.
• To improve the utility of synthetic text data while ensuring formal privacy protection.
• To address the limitations of existing DP methods in large-scale text generation.

Main Methods:

• Leveraging large language models (LLMs) for generating high-quality synthetic samples.
• Employing a DP-based discriminator, constructed via knowledge distillation, to select domain-fitting samples.
• Utilizing a DP-based tutor to align label distributions with the private domain under a low privacy budget.

Main Results:

• The proposed DP-based DA framework effectively generates high-quality, privacy-preserving text data.
• DP-synthesized samples significantly outperform state-of-the-art DP fine-tuning baselines in utility.
• Empirical validation on three medical text classification datasets demonstrates superior performance.

Conclusions:

• The novel DP-based DA framework offers a robust solution for privacy-preserving text generation in sensitive domains.
• This approach successfully balances data utility and formal privacy guarantees, outperforming existing methods.
• The method shows significant promise for applications requiring secure and effective data augmentation.