Bridging the Gap: Rademacher Complexity in Robust and Standard Generalization.

Area of Science:

• Machine Learning
• Deep Learning Theory
• Adversarial Robustness

Background:

• Training Deep Neural Networks (DNNs) with adversarial examples often leads to poor generalization on adversarial data.
• Existing research on adversarially robust generalization faces challenges in achieving satisfactory bounds, often yielding looser results than standard generalization.

Purpose of the Study:

• To investigate adversarially robust generalization in DNNs using Rademacher complexity.
• To derive tighter upper bounds for adversarial Rademacher complexity that match standard generalization bounds.

Main Methods:

• The study employs Rademacher complexity to analyze adversarial generalization.
• A novel concept, the 'uniform covering number', is introduced to calculate the covering number of adversarial function classes.
• The proposed uniform covering number is designed for compatibility with adversarial examples and precision comparable to standard settings.

Main Results:

• The paper establishes new upper bounds for the adversarial Rademacher complexity of DNNs.
• These bounds match the best-known upper bounds in standard settings, with a dependency of $$ on width and dimension.
• The proposed method successfully bridges the gap between Rademacher complexity in robust and standard generalization.

Conclusions:

• The introduction of the uniform covering number provides a significant advancement in understanding and improving adversarial robustness in DNNs.
• The derived bounds suggest that DNNs can achieve robust generalization comparable to standard generalization under certain conditions.
• This work offers a theoretical foundation for developing more robust Deep Neural Networks against adversarial attacks.