Dataset Condensation via Expert Subspace Projection.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep learning models require large datasets, increasing storage and training costs.
• Dataset condensation aims to create smaller synthetic datasets that retain performance.
• Existing trajectory matching methods are computationally expensive due to gradient unrolling.

Purpose of the Study:

• To propose a novel dataset condensation method, Expert Subspace Projection (ESP).
• To reduce computational and memory overhead associated with trajectory matching.
• To maintain downstream model performance with significantly smaller datasets.

Main Methods:

• Expert Subspace Projection (ESP) constrains synthetic data trajectories within the real data's trajectory subspace.
• Avoids complex gradient unrolling across multiple training iterations.
• Enables unbiased training on the full synthetic dataset.

Main Results:

• ESP significantly outperforms trajectory matching methods, achieving 16.7% improvement on CIFAR10 (1 Image/Class).
• The proposed method surpasses the previous state-of-the-art by 3.2%.
• ESP offers memory-saving advantages and integrates with other condensation techniques.

Conclusions:

• Expert Subspace Projection (ESP) is an effective dataset condensation technique.
• ESP reduces computational costs while maintaining or improving model performance.
• This method addresses the urgent need for efficient dataset size reduction in deep learning.