Fast yet versatile machine unlearning for deep neural networks

In response to the growing concerns regarding data privacy, many countries and organizations have implemented corresponding laws and regulations, such as the General Data Protection Regulation (GDPR), to safeguard users' data privacy. Among these, the Right to Be Forgotten holds particular significance, signifying the necessity for data to be forgotten from improper use. Recently, researchers have integrated the concept of the Right to Be Forgotten into the field of machine learning, focusing on the unlearning of data from machine learning models. However, existing studies either require additional storage for caching updates during the model training phase or are only applicable in specific forgotten scenarios. In this paper, we propose a versatile unlearning method that involves unlearning data by fine-tuning the model until the distribution of the model's prediction for the forgotten data matches those for unseen third-party data. Importantly, our method does not require additional storage for caching model updates, and it can be applied across different forgotten scenarios. Experimental results demonstrate the efficacy of our method in unlearning backdoor triggers, entire classes of training data, and subsets of training data.