Advancing Pre-Trained Teacher: Towards Robust Feature Discrepancy for Anomaly Detection

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Unsupervised anomaly detection commonly uses knowledge distillation between teacher and student models.
• Success relies on maintaining feature discrepancy, assuming separable normal/abnormal distributions for the teacher and normal-only reconstruction for the student.
• Practical implementation faces challenges in upholding these assumptions.

Purpose of the Study:

• To propose a novel two-stage industrial anomaly detection framework, AAND.
• To enhance the core assumptions of knowledge distillation for improved anomaly detection.
• To achieve state-of-the-art performance in unsupervised anomaly detection tasks.

Main Methods:

• AAND employs a two-stage approach: Anomaly Amplification and Normality Distillation.
• The Anomaly Amplification stage uses a Residual Anomaly Amplification (RAA) module with synthetic anomalies to boost the teacher model.
• The Normality Distillation stage utilizes a reverse distillation paradigm and a Hard Knowledge Distillation (HKD) loss for the student decoder.

Main Results:

• The proposed AAND framework demonstrates state-of-the-art performance.
• Experiments were conducted on MvTecAD, VisA, and MvTec3D-RGB datasets.
• The RAA module effectively amplifies anomalies while preserving feature integrity.

Conclusions:

• The AAND framework successfully enhances the assumptions critical for knowledge distillation in anomaly detection.
• The proposed methods, RAA and HKD loss, contribute to superior performance in industrial anomaly detection.
• The framework offers a simple yet effective solution for complex anomaly detection challenges.