Avoiding catastrophic overfitting in fast adversarial training with adaptive similarity step size

Area of Science:

• Deep Learning
• Machine Learning Security
• Computer Vision

Background:

• Adversarial training is crucial for deep learning model robustness.
• Fast adversarial training methods offer computational efficiency but suffer from limited adversarial example diversity, leading to overfitting and reduced robustness.
• Existing methods to mitigate overfitting have limitations in adversarial example strength and overall perturbation.

Purpose of the Study:

• To propose a novel fast adversarial training method, ATSS (Adaptive Similarity Step Size), to address the limitations of current approaches.
• To enhance the robustness and accuracy of deep learning models against adversarial attacks while preventing catastrophic overfitting.

Main Methods:

• ATSS involves adding random noise to clean samples and calculating gradients.
• Perturbation step size is adaptively determined based on the similarity between input noise and gradient direction.
• Adversarial examples are generated using the adaptive step size for training.

Main Results:

• Experimental results on ResNet18 and VGG19 models across CIFAR-10, CIFAR-100, and Tiny ImageNet datasets demonstrate ATSS's effectiveness.
• The proposed ATSS method successfully avoids catastrophic overfitting.
• ATSS achieves superior robustness and clean accuracy compared to existing fast adversarial training methods with minimal additional training cost.

Conclusions:

• ATSS offers an effective solution for fast adversarial training, balancing robustness and accuracy.
• The adaptive similarity step size mechanism is key to generating diverse and effective adversarial examples.
• This method presents a computationally efficient approach to significantly improve deep learning model resilience.