InfoAT: Improving Adversarial Training Using the Information Bottleneck Principle.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Adversarial training (AT) is effective for defending against adversarial examples.
• Not all examples contribute equally to model robustness during AT; hard examples are more influential.
• Identifying effective heuristics for finding hard examples remains a challenge.

Purpose of the Study:

• To propose a novel adversarial training method that enhances model robustness by focusing on influential examples.
• To leverage the information bottleneck principle to identify examples that are more susceptible to adversarial attacks.

Main Methods:

• Inspired by the information bottleneck (IB) principle, we identify examples with high mutual information between input and latent representation as likely targets for attack.
• Proposed a novel adversarial training method, InfoAT, designed to find and exploit these high mutual information examples.
• Evaluated InfoAT against state-of-the-art methods on various datasets and models.

Main Results:

• InfoAT effectively identifies examples with high mutual information, which are more likely to be attacked.
• The proposed InfoAT method demonstrably improves the final robustness of models.
• Experimental results show InfoAT achieving superior robustness compared to existing methods.

Conclusions:

• Focusing on hard examples, identified via mutual information, is crucial for improving adversarial robustness.
• InfoAT provides an effective strategy for enhancing model defense against adversarial attacks.
• The information bottleneck principle offers a valuable perspective for developing robust machine learning models.