An Adaptive Black-Box Defense Against Trojan Attacks (TrojDef).

Area of Science:

• Artificial Intelligence
• Machine Learning Security
• Cybersecurity

Background:

• Neural networks (NNs) are vulnerable to Trojan backdoor attacks, where adversaries implant malicious functionalities via poisoned training data.
• Existing defenses often require white-box access, limiting their practical applicability.
• Black-box attacks, where only forward passes are accessible, pose a significant threat.

Purpose of the Study:

• To propose TrojDef, a novel black-box defense mechanism against Trojan backdoor attacks in NN classifiers.
• To develop a method for identifying and filtering Trojan-infected inputs without access to model internals.
• To enhance the robustness and practical applicability of defenses against sophisticated poisoning attacks.

Main Methods:

• TrojDef monitors prediction confidence changes when inputs are repeatedly perturbed with random noise.
• A 'prediction confidence bound' is derived from output predictions to distinguish benign from Trojan inputs.
• A nonlinear transform is applied to the confidence bound to improve detection accuracy in imperfect attack scenarios.

Main Results:

• TrojDef effectively identifies Trojan inputs by exploiting their stability under noise perturbation.
• The proposed defense significantly outperforms state-of-the-art black-box defenses.
• TrojDef demonstrates high stability across various classifier architectures, training processes, and hyperparameters.

Conclusions:

• TrojDef provides a practical and effective black-box defense against NN Trojan backdoor attacks.
• The method's reliance on prediction confidence under noise offers a robust detection strategy.
• This research advances the field of adversarial machine learning by offering a viable solution for real-world security concerns.