Hierarchical Knowledge Transfer: Cross-Layer Distillation for Industrial Anomaly Detection

Area of Science:

• Computer Vision
• Machine Learning
• Industrial Quality Control

Background:

• Traditional knowledge distillation for industrial anomaly detection faces limitations in feature alignment and model structure, hindering multi-faceted anomaly representation.
• Existing methods often rely on same-layer feature alignment and similar teacher-student architectures, restricting detection capabilities.

Purpose of the Study:

• To propose a novel Hierarchical Knowledge Transfer (HKT) framework to enhance industrial surface anomaly detection.
• To address limitations in cross-layer interaction and structural symmetry in knowledge distillation for anomaly detection.

Main Methods:

• HKT framework utilizes deep knowledge from the teacher's highest feature layer to guide student learning across all levels, facilitating cross-layer interactions.
• Multiple projectors enable knowledge transfer from the teacher to each student layer.
• Convolutional Block Attention Modules (CBAM) are integrated into the student network to decouple teacher-student structural symmetry.
• An enhanced model, HKT+, is developed by adding convolutional layers to HKT for improved detection.

Main Results:

• HKT+ achieves state-of-the-art performance on the MVTec AD and BeanTech AD (BTAD) datasets.
• Average AUROC scores of 98.69% on MVTec AD and 94.58% on BTAD were recorded.
• The HKT+ model demonstrates superior anomaly detection capabilities with a minimal increase in parameters.

Conclusions:

• The proposed HKT framework effectively enhances industrial anomaly detection by enabling cross-layer knowledge transfer and structural decoupling.
• HKT+ represents a significant advancement in anomaly detection, outperforming existing state-of-the-art methods.
• The framework offers a more robust and efficient approach to identifying surface anomalies in industrial settings.