Relation Knowledge Distillation by Auxiliary Learning for Object Detection.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Object detection demands a balance between accuracy and inference speed.
• Knowledge distillation offers a model compression technique to transfer knowledge from complex teacher models to simpler student models.
• Existing distillation methods for object detection often adapt image classification techniques and overlook the intrinsic relationship between classification and localization predictions.

Purpose of the Study:

• To propose a novel method, Relation Knowledge Distillation by Auxiliary Learning (ReAL), for object detection.
• To address limitations in current knowledge distillation methods for object detection.
• To improve the efficiency and effectiveness of object detection models.

Main Methods:

• Designed a prediction relation distillation module for direct imitation of teacher model outputs.
• Implemented self and mutual relation distillation losses to capture inter-model relationships.
• Introduced auxiliary learning with a dynamic weight adaptation strategy, treating detection as the primary task and distillation as auxiliary.

Main Results:

• ReAL demonstrated significant improvements across various teacher-student model configurations on the MS COCO dataset.
• The method effectively balances accuracy and speed in object detection.
• Experimental results show favorable performance compared to state-of-the-art methods.

Conclusions:

• ReAL offers an effective approach to knowledge distillation for object detection.
• The proposed method successfully addresses the trade-off between accuracy and inference time.
• ReAL provides a promising direction for developing efficient and accurate object detection systems.