Combating Overfitting of Adversarial Training Efficiently via Balanced Instance Adaptive Defense

Adversarial training, one of the most effective methods for enhancing neural network robustness, is typically formulated as a min-max game between an attacker and a defender. Despite its success, most adversarial training methods suffer from robust overfitting, leading to a significant gap in robustness between the training and test sets. In this paper, we theoretically analyze this phenomenon through the lens of feature learning. Our analysis reveals that fitting non-robust features is the root cause of overfitting, which is exacerbated by the attacker and defender with unbalanced optimization strength. Specifically, a defender that is too strong forces the model to learn non-robust features, while an attacker that is too strong hinders the feature learning process and leads to extra computational costs. To address this issue, we propose Balanced Instance Adaptive Defense (BIAD), which dynamically balances attackers and defenders at the instance level. BIAD classifies training examples into robust and non-robust groups and strengthens the attacker and regularization for robust examples while weakening them for non-robust ones. Experimental results across various architectures and datasets (CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet) demonstrate that BIAD achieves higher training efficiency, better accuracy-robustness trade-offs, and effectively mitigates robust overfitting.