Multilayer Semantic Features Adaptive Distillation for Object Detectors.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Knowledge distillation (KD) is crucial for compressing neural networks, particularly in object detection.
• Existing KD methods often use fixed semantic features, limiting adaptability across training stages and samples.

Purpose of the Study:

• To propose a novel multilayer semantic feature adaptive distillation (MSFAD) method for object detection.
• To enhance the efficiency and effectiveness of knowledge distillation in training student object detectors.

Main Methods:

• Developed a routing network with teacher and student detectors and an agent network for decision-making.
• Utilized a proxy network that processes features from teacher and student neck structures to select optimal features for distillation.
• Implemented an adaptive selection mechanism for valuable semantic-level features from the teacher to the student detector.

Main Results:

• The MSFAD method significantly improved object detection performance.
• Achieved a 3.4% increase in mAP50 and a 3.3% increase in mAP50-90 for YOLOv5s.
• YOLOv5n, despite having only 1.9M parameters, demonstrated detection performance comparable to YOLOv5s.

Conclusions:

• MSFAD offers an adaptive approach to feature selection in knowledge distillation for object detection.
• The proposed method enhances student model performance and enables efficient compression.
• Results indicate potential for developing highly performant, lightweight object detection models.