When Object Detection Meets Knowledge Distillation: A Survey.

Area of Science:

• Computer Vision
• Machine Learning

Background:

• Object detection (OD) models have advanced but increased complexity hinders practical application.
• Knowledge distillation (KD) enables transferring knowledge from large teacher models to smaller, efficient student models.

Purpose of the Study:

• To provide a comprehensive survey of recent knowledge distillation-based object detection (KD-OD) models.
• To analyze the advantages, limitations, and future research directions in KD-OD.

Main Methods:

• In-depth analysis of existing KD-OD works.
• Categorization of KD-OD tasks, including lightweight model improvement, incremental OD, small object detection, and weakly/semi-supervised OD.
• Analysis of distillation techniques like distillation loss and feature interaction.

Main Results:

• Overview of KD-OD model principles and applications on datasets like remote sensing and 3D point clouds.
• Comparison and performance analysis of various KD-OD models.
• Identification of challenges and opportunities in KD-OD.

Conclusions:

• KD is a vital technique for developing efficient OD models.
• Further research is needed to address specific OD challenges and expand applications.
• This survey offers insights for designing practical and effective KD-OD systems.