Structured Knowledge Distillation for Accurate and Efficient Object Detection.

Area of Science:

• Computer Vision
• Machine Learning

Background:

• Knowledge distillation is effective for image classification but struggles with object detection.
• Existing methods fail due to foreground-background pixel imbalance and lack of relational knowledge transfer.

Purpose of the Study:

• To propose a structured knowledge distillation scheme for object detection.
• To address the limitations of previous distillation methods in complex vision tasks.

Main Methods:

• Introduced attention-guided distillation to focus on crucial foreground object pixels.
• Implemented non-local distillation to capture relationships between different pixels.

Main Results:

• Demonstrated effectiveness across thirteen object detection models and twelve comparison methods.
• Achieved 43.9 mAP on MS COCO2017 with Faster RCNN, a 4.1 mAP improvement.
• Showcased benefits for model robustness and domain generalization.

Conclusions:

• The proposed structured knowledge distillation effectively enhances object detection and instance segmentation.
• The method successfully transfers knowledge by addressing pixel imbalance and relational information.
• The approach improves model robustness and generalization capabilities.