When Not to Classify: Anomaly Detection of Attacks (ADA) on DNN Classifiers at Test Time

Area of Science:

• Machine Learning Security
• Deep Neural Networks
• Adversarial Attacks

Background:

• Machine learning systems, particularly deep neural networks (DNNs), face significant threats from adversarial learning attacks, specifically evasion attacks at test time.
• Existing defenses often focus on robustifying classifiers against perturbed inputs, but this is not always optimal or feasible.
• In certain scenarios, detecting an attack is more valuable than correctly classifying a perturbed input, especially when the attacker is the sole recipient of the decision.

Purpose of the Study:

• To propose and evaluate a novel unsupervised anomaly detection (AD) method for identifying adversarial perturbations in deep neural networks.
• To demonstrate that adversarial perturbations are machine detectable, even when small.
• To offer a detection-based defense strategy that is actionable in specific adversarial scenarios.

Main Methods:

• Developed a purely unsupervised anomaly detector (AD) that models joint densities of deep layers using null hypothesis density models.
• Incorporated multiple DNN layers, source/destination class concepts, class confusion matrices, and DNN weight information.
• Constructed a novel decision statistic based on Kullback-Leibler divergence for attack detection.

Main Results:

• The proposed AD method outperformed previous detection methods on MNIST and CIFAR image databases under three attack strategies.
• Achieved strong Receiver Operating Characteristic (ROC) area under the curve (AUC) detection accuracy for two attacks.
• Demonstrated superior accuracy compared to recent methods on the strongest (CW) attack and effectiveness against white-box reverse engineering attacks.

Conclusions:

• Adversarial perturbations are detectable using unsupervised anomaly detection, even subtle ones.
• The novel AD method provides a robust and effective defense against various adversarial evasion attacks.
• The approach offers a valuable alternative or complement to classifier robustification, particularly in scenarios where attack detection is paramount.