Knowledge Distillation in Video-Based Human Action Recognition: An Intuitive Approach to Efficient and Flexible Model...

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Training human action recognition models in videos is computationally demanding.
• Current transfer learning methods lack flexibility and efficiency, often relying on restrictive pretrained architectures.

Purpose of the Study:

• To explore knowledge distillation (KD) for enhancing self-supervised video model training.
• To improve classification accuracy, accelerate convergence, and increase model flexibility.
• To evaluate KD's effectiveness in regular and limited-data scenarios.

Main Methods:

• Applied knowledge distillation (KD) to guide the training of self-supervised video models.
• Tested the method on the UCF101 dataset with varying data proportions (100%, 50%, 25%, 2%).
• Compared KD-guided training against traditional training methods.

Main Results:

• Knowledge distillation outperformed traditional training without compromising classification accuracy.
• KD reduced the convergence rate in both standard and data-scarce environments.
• Enabled cross-architecture flexibility for diverse applications, from resource-limited to high-performance.

Conclusions:

• Knowledge distillation is an effective technique for improving the efficiency and flexibility of self-supervised video model training.
• KD offers a viable solution for scenarios with limited data.
• The method allows for adaptable model customization across different computational constraints.