"Understanding Robustness Lottery": A Geometric Visual Comparative Analysis of Neural Network Pruning Approaches.

Area of Science:

• Deep Learning
• Computer Vision
• Machine Learning

Background:

• Deep learning models achieve high performance but are often large and brittle.
• Model pruning is a key technique to create smaller, more robust networks.
• Current understanding of pruning's impact on internal representations is limited.

Purpose of the Study:

• To investigate how different pruning methods alter neural network feature representations.
• To analyze the impact of these alterations on model performance and robustness.
• To develop a visualization tool for comparing pruning strategies.

Main Methods:

• Introduced a visual geometric analysis of high-dimensional feature representations.
• Evaluated geometric concepts derived from classification loss.
• Designed a visualization system to compare pruning impacts.

Main Results:

• The visualization system effectively highlights the effects of pruning on feature representations.
• Geometric analysis reveals differences in feature space geometry between pruned models.
• Identified similarities between pruning methods and potential redundancy in robustness benchmarks.

Conclusions:

• The developed visualization tool offers insights into pruning's effect on model behavior.
• Researchers can use this tool to compare pruning methods and understand model robustness.
• Facilitates identification of robust/fragile samples under pruning and data corruption.