DD-RobustBench: An Adversarial Robustness Benchmark for Dataset Distillation.

Area of Science:

• Machine Learning
• Computer Vision
• Data Science

Background:

• Dataset distillation offers efficient data utilization by creating smaller, accurate subsets.
• Current methods reduce data size and training time but lack robust evaluation against adversarial attacks.
• This vulnerability poses risks in security-sensitive AI applications.

Purpose of the Study:

• Introduce DD-RobustBench, a comprehensive benchmark for evaluating adversarial robustness of distilled datasets.
• Assess the impact of various dataset distillation techniques on robustness.
• Provide insights for developing more secure and resilient distilled datasets.

Main Methods:

• Integrated diverse dataset distillation techniques (TESLA, DREAM, SRe2L, D4M).
• Tested distilled datasets against a wide range of adversarial attack methods.
• Evaluated robustness across multiple datasets, including ImageNet-1K.
• Quantitatively analyzed the influence of distillation components (augmentation, downsampling, clustering) on robustness.

Main Results:

• Identified specific distillation techniques and components that enhance or degrade adversarial robustness.
• Demonstrated the vulnerability of current distilled datasets to adversarial threats.
• Provided quantitative evidence on the trade-offs between data compression and robustness.

Conclusions:

• DD-RobustBench serves as a critical tool for assessing and improving the security of distilled datasets.
• Findings offer guidelines for creating compact, efficient, and robust distilled datasets.
• Highlights the need for further research into secure dataset distillation methods.