Dataset Distillation via a Noise-Unconstrained Generative Model

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Dataset distillation (DD) aims to create smaller datasets that retain the performance of original, larger datasets.
• Existing generative model (GM)-based DD methods struggle with sample representativeness and inter-sample relationships.

Purpose of the Study:

• To propose a novel noise-unconstrained generative model-based framework for dataset distillation.
• To enhance the effectiveness of DD by addressing limitations in sample generation and optimization.

Main Methods:

• Introduced an adaptive matching coefficient for aligning generated images with class representatives.
• Extended the MiniMax loss function to simplify optimization.
• Employed gradient-matching based DD for ensembling features among generative images.

Main Results:

• Theoretical analysis using McDiarmid's inequality supports reduced generalization error.
• Generated images demonstrated superior performance as proxy datasets on ImageWoof.
• Achieved significant performance gains over baseline methods with fewer distilled images.

Conclusions:

• The proposed framework offers an effective approach to dataset distillation using generative models.
• Generated images show promise as efficient proxies for training deep learning models.
• The method is robust across various dataset resolutions and benchmarks.