Prevalence of simplex compression in adversarial deep neural networks

Area of Science:

• Deep Learning Theory
• Machine Learning Robustness
• Neural Network Analysis

Background:

• Neural collapse (NC) describes a phenomenon where neural network outputs for intra-class samples converge, while inter-class samples form a simplex equiangular tight frame (ETF).
• Understanding the intrinsic properties and geometric structures within neural networks is crucial for improving their performance and robustness.

Purpose of the Study:

• To investigate the impact of adversarial training on the simplex ETF structure observed in neural collapse.
• To identify and characterize a potential 'simplex compression' phenomenon under adversarial conditions.
• To develop a theoretical framework explaining the observed geometric changes in neural representations.

Main Methods:

• Empirical analysis across diverse models and datasets to observe the geometric size of the simplex ETF.
• Systematic application of adversarial training with varying perturbation radii.
• Development of a theoretical framework to explain the observed simplex compression phenomenon.

Main Results:

• A novel 'simplex compression' phenomenon was identified in neural collapse under adversarial training.
• The geometric size of the simplex ETF demonstrably reduces with adversarial training.
• The degree of simplex compression correlates positively with the perturbation radius of adversarial attacks.

Conclusions:

• Adversarial training induces a compression of the geometric structure within neural representations, as evidenced by neural collapse.
• The findings provide a deeper understanding of neural network behavior under adversarial conditions.
• The established theoretical framework offers insights into the robustness of neural networks and the mechanisms of neural collapse.