Towards improving fast adversarial training in multi-exit network.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Adversarial examples pose a significant threat to machine learning models, aiming to cause misclassifications.
• Projected Gradient Descent (PGD) is a common method for enhancing adversarial robustness but is computationally expensive.
• Fast adversarial training offers speed but often results in suboptimal performance due to limited training epochs.

Purpose of the Study:

• To investigate the potential of multi-exit networks in improving adversarial robustness.
• To address the time-consuming nature of traditional adversarial training methods.
• To mitigate catastrophic overfitting observed in single-step adversarial training.

Main Methods:

• Utilizing multi-exit networks to process adversarial examples, allowing early exits for simpler classifications.
• Analyzing weight norms in different exits to understand catastrophic overfitting dynamics.
• Proposing a novel approach to alleviate catastrophic overfitting in multi-exit networks.

Main Results:

• Multi-exit networks effectively reduce the impact of adversarial perturbations by leveraging early exits.
• Catastrophic overfitting in multi-exit networks is linked to specific weight norm distributions across exits.
• The proposed approach successfully alleviates catastrophic overfitting, leading to enhanced empirical robustness.

Conclusions:

• Multi-exit networks offer a promising direction for improving adversarial robustness and training efficiency.
• Understanding weight norm behavior is crucial for preventing overfitting in multi-exit architectures.
• The developed method achieves competitive or superior robustness compared to PGD adversarial training with reduced time complexity.