Localization Distillation for Object Detection.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Knowledge distillation (KD) for object detection typically relies on feature imitation.
• Mimicking prediction logits is often overlooked due to perceived inefficiency in transferring localization information.

Purpose of the Study:

• To investigate if logit mimicking can outperform feature imitation in object detection.
• To develop an efficient method for transferring localization knowledge from teacher to student models.

Main Methods:

• Introduced a novel localization distillation (LD) method for efficient knowledge transfer.
• Proposed the concept of valuable localization regions for selective knowledge distillation.
• Combined LD with valuable localization regions to enhance logit mimicking.

Main Results:

• Demonstrated that logit mimicking, when enhanced with localization distillation, surpasses feature imitation.
• Showcased significant improvements in alleviating localization ambiguity and learning robust features.
• Validated the method's effectiveness on MS COCO, PASCAL VOC, and DOTA benchmarks, achieving considerable AP gains.

Conclusions:

• The absence of localization distillation was a key reason for logit mimicking's previous underperformance.
• Logit mimicking, with the proposed LD, offers a simple yet effective distillation scheme.
• The method is applicable to both dense horizontal and rotated object detectors, improving performance without inference speed sacrifice.