Practical Continual Forgetting for Pre-Trained Vision Models

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Growing privacy and security concerns necessitate the removal of specific information from pre-trained vision models.
• Real-world erasure requests are sequential, requiring continuous selective information removal while preserving existing knowledge.
• The problem of continual forgetting presents challenges in efficient deletion, minimal impact on remaining knowledge, and handling scarce or missing training data.

Purpose of the Study:

• To address the challenges of continual forgetting in pre-trained vision models.
• To develop a method for selective information erasure that is efficient and effective.
• To minimize the impact of information removal on the model's retained knowledge.

Main Methods:

• Proposes Group Sparse LoRA (GS-LoRA) with independent fine-tuning modules for each forgetting task.
• Employs group sparse regularization to automatically select and zero out specific LoRA groups.
• Introduces GS-LoRA++ incorporating prototype information for enhanced class-specific forgetting and retention.

Main Results:

• Demonstrates effective forgetting of specific classes in vision models.
• Shows minimal impact on the performance of remaining classes after information erasure.
• Validated through extensive experiments on face recognition, object detection, and image classification tasks.

Conclusions:

• GS-LoRA and GS-LoRA++ provide efficient and effective solutions for continual forgetting in pre-trained vision models.
• The methods successfully balance the erasure of unwanted information with the preservation of desired knowledge.
• The approach is practical for real-world scenarios with potential data scarcity.