Contrastive learning unlocks geometric insights for dataset pruning.

Area of Science:

• Computer Science
• Machine Learning
• Data Science

Background:

• Dataset pruning is crucial for managing big data, especially in unsupervised settings to avoid expensive labeling.
• Existing unsupervised methods often treat representation learning as a black box, with limited exploration of embedding properties for pruning.

Purpose of the Study:

• To develop an effective unsupervised dataset pruning strategy by analyzing the geometric properties of learned embedding manifolds.
• To leverage contrastive learning's embedding space for efficient data selection.

Main Methods:

• Revisiting self-supervised contrastive learning by observing the learned embedding manifold.
• Introducing Curvature Estimation to characterize manifold geometry.
• Proposing KITTY sampling: an unsupervised strategy involving downsampling in high instance density areas.

Main Results:

• Statistical analysis revealed non-uniform instance distribution on embedding manifold surfaces.
• KITTY sampling demonstrated leading performance in computer vision dataset pruning tasks compared to baseline methods.
• The proposed method effectively prunes datasets without compromising model performance.

Conclusions:

• Unsupervised dataset pruning can be significantly enhanced by understanding the geometric structure of embedding manifolds.
• KITTY sampling offers an efficient and effective approach to dataset pruning in the era of big data.
• The study highlights the potential of geometric analysis in representation learning for data management tasks.