Unlearning Comparator: A Visual Analytics System for Comparative Evaluation of Machine Unlearning Methods.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Privacy

Background:

• Machine Unlearning (MU) is crucial for data privacy, enabling the removal of specific training data from models.
• Current evaluation of MU methods lacks systematic analysis, hindering understanding of accuracy, efficiency, and privacy trade-offs.
• Researchers struggle with aggregate metrics and ad-hoc evaluations for assessing diverse MU techniques.

Purpose of the Study:

• Introduce a visual analytics system, Unlearning Comparator, for systematic evaluation of Machine Unlearning methods.
• Facilitate in-depth analysis of MU method behavior at class, instance, and layer levels.
• Enable robust privacy assessment through simulated membership inference attacks.

Main Methods:

• Developed Unlearning Comparator, a visual analytics system for evaluating MU methods.
• Implemented model comparison features for class-, instance-, and layer-level analysis.
• Integrated membership inference attack (MIA) simulations for privacy evaluation.

Main Results:

• The Unlearning Comparator system allows for detailed comparison between unlearned models and baselines.
• Visual analysis of prominent MU methods revealed insights into model behavior post-unlearning.
• The system effectively aids in understanding and improving Machine Unlearning techniques.

Conclusions:

• Unlearning Comparator provides a systematic framework for evaluating Machine Unlearning methods.
• The system enhances the understanding of accuracy, efficiency, and privacy trade-offs in MU.
• Visual analytics and attack simulations are vital for advancing Machine Unlearning research.