Efficient Crowd Counting via Dual Knowledge Distillation.

Area of Science:

• Computer Vision
• Artificial Intelligence

Background:

• Current crowd counting models prioritize accuracy over deployment efficiency, leading to high computational costs.
• Knowledge distillation offers a solution by transferring knowledge from large teacher models to smaller student models, but can suffer from inaccurate teacher guidance.

Purpose of the Study:

• To propose a Dual-Knowledge Distillation (DKD) framework for efficient crowd counting.
• To mitigate negative impacts from teacher models and transfer hierarchical knowledge for improved efficiency.

Main Methods:

• DKD initializes student models with global information from teacher models using adaptive perspectives.
• Self-knowledge distillation guides student learning using intermediate feature maps and target maps.
• Optimal transport distance is employed for feature map distribution alignment between teacher and student models.

Main Results:

• The DKD framework significantly enhances crowd counting efficiency and accuracy.
• Student models achieve comparable or superior performance to teacher models with only ~6% of their parameters and computations.
• Experiments on four datasets validate the superiority of the proposed DKD approach.

Conclusions:

• DKD effectively transfers knowledge for efficient and accurate crowd counting.
• The framework addresses limitations of standard knowledge distillation by reducing teacher-induced errors.
• DKD presents a promising solution for deploying high-performance crowd counting models in resource-constrained environments.