Feature Map Distillation of Thin Nets for Low-Resolution Object Recognition.

Area of Science:

• Computer Vision
• Artificial Intelligence

Background:

• Intelligent video surveillance in natural environments faces challenges with low-resolution and noisy object recognition.
• Existing knowledge distillation methods often reduce network channels, neglecting feature map size, thus limiting knowledge transfer from deep teacher networks to shallow student networks.

Purpose of the Study:

• To propose a novel Feature Map Distillation (FMD) framework for enhanced knowledge transfer in computer vision.
• To address the limitations of existing methods by focusing on feature map size differences between teacher and student networks.

Main Methods:

• Feature Map Distillation (FMD) framework with two components: Feature Decoder Distillation (FDD) and Feature Map Consistency-enforcement (FMC).
• FDD reconstructs shallow features of the student network to match teacher network samples, enabling high-resolution guidance.
• FMC ensures consistent feature map size and direction between networks, enforcing similar feature distributions.

Main Results:

• The FMD framework enables thin student networks to learn "privilege information" from wide teacher networks.
• Experimental validation on multiple recognition tasks demonstrates superior performance compared to state-of-the-art knowledge distillation methods.
• The method is effective for recognizing low-resolution and noisy objects in surveillance scenarios.

Conclusions:

• Feature Map Distillation (FMD) offers a significant advancement in knowledge distillation for computer vision.
• The proposed FDD and FMC components effectively bridge the gap in feature representation between networks of different sizes.
• FMD enhances the performance of student networks in challenging surveillance applications with low-resolution and noisy data.