Deep Unlearning via Randomized Conditionally Independent Hessians.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Privacy

Background:

• Machine unlearning is increasingly important due to privacy regulations and data quality concerns.
• Existing methods for unlearning in complex models are computationally expensive, often requiring Hessian matrix inversion.
• Simpler models like k-NN can be unlearned by data deletion, but this is not applicable to deep learning models.

Purpose of the Study:

• To develop an efficient and scalable method for machine unlearning in complex predictive models.
• To address the computational limitations of existing optimization-based unlearning techniques.
• To enable approximate unlearning for deep learning models in domains like computer vision and natural language processing.

Main Methods:

• Utilized a variant of the L-CODEC (conditional independence coefficient) to identify model parameters with high semantic overlap for individual samples.
• Employed Markov blanket selection to pinpoint relevant parameters for targeted unlearning.
• Avoided the need for computationally intensive Hessian matrix inversion.

Main Results:

• The L-CODEC approach successfully identifies a subset of model parameters for efficient unlearning.
• The method scales effectively with model dimension, overcoming limitations of previous techniques.
• Demonstrated feasibility of approximate unlearning for deep vision and NLP models.

Conclusions:

• L-CODEC offers a computationally feasible solution for machine unlearning in deep learning models.
• This method facilitates unlearning in sensitive applications such as face recognition, person re-identification, and NLP.
• The approach makes approximate unlearning viable in previously infeasible scenarios.