DP2Unlearning: An efficient and guaranteed unlearning framework for LLMs.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Privacy

Background:

• Large language models (LLMs) excel at language tasks but raise ethical concerns regarding memorization of private or copyrighted data.
• Retraining LLMs to remove specific data is effective but computationally prohibitive.
• Existing approximate unlearning methods lack formal forgetting guarantees.

Purpose of the Study:

• Introduce DP2Unlearning, a novel framework for efficient LLM unlearning with formal forgetting guarantees.
• Provide a cost-effective solution for removing sensitive information from LLMs compared to complete retraining.
• Ensure privacy against data disclosure while maintaining model performance.

Main Methods:

• Train LLMs on textual data protected with epsilon-differential privacy (DP).
• Utilize the DP-protected model to enable efficient unlearning with formal privacy guarantees.
• Compare DP2Unlearning against exact retraining and approximate unlearning methods.

Main Results:

• DP2Unlearning achieves comparable model performance to exact unlearning (retraining from scratch) post-unlearning.
• The proposed method offers unlearning at approximately half the computational cost of retraining.
• DP2Unlearning outperforms approximate unlearning methods in preserving model utility and effectively forgetting targeted data.

Conclusions:

• DP2Unlearning provides a practical and theoretically sound approach to LLM unlearning.
• The framework balances data privacy, model utility, and computational efficiency.
• This method offers a viable solution for addressing ethical and legal challenges in LLM deployment.