Backdoor samples detection based on perturbation discrepancy consistency in pre-trained language models.

Area of Science:

• Artificial Intelligence
• Machine Learning Security

Background:

• Pre-trained models are vulnerable to backdoor attacks from unvetted data.
• Existing detection methods are often impractical due to resource or access requirements.

Purpose of the Study:

• To develop a practical and effective backdoor sample detection method.
• To enable detection in both pre-training and post-training phases.

Main Methods:

• Propose Perturbation Discrepancy Consistency Evaluation (NETE).
• Utilize off-the-shelf pre-trained models and a mask-filling strategy for perturbations.
• Measure log probability discrepancies using curvature to evaluate consistency.

Main Results:

• NETE leverages the phenomenon that perturbation discrepancy changes less for backdoor samples than clean samples.
• The method outperforms existing zero-shot black-box detection techniques.
• Demonstrated effectiveness against four typical backdoor attacks and five large language model backdoor attack types.

Conclusions:

• NETE offers a practical solution for detecting backdoor samples.
• The method is effective across various attack types and model phases.
• Advances the security of pre-trained models against data poisoning.