Spatiotemporal Knowledge Distillation for Efficient Estimation of Aerial Video Saliency

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Deep learning, particularly Convolutional Neural Networks (CNNs), has advanced video saliency estimation.
• Complex deep saliency models are challenging to deploy on resource-constrained devices like cameras and drones due to computational and storage limitations.

Purpose of the Study:

• To propose an efficient dynamic saliency estimation approach for aerial videos.
• To enable the deployment of saliency estimation models on devices with limited computational capabilities.

Main Methods:

• A spatiotemporal knowledge distillation approach involving two teacher models, two student models, and a unified spatiotemporal model.
• Transferring spatial and temporal saliency knowledge from complex teachers to compact students.
• Degrading input resolution to reduce data redundancy and accelerate feature extraction.
• Training the unified spatiotemporal model by distilling knowledge from the student models to remove inter-model redundancy.

Main Results:

• The proposed approach achieves performance comparable to 11 state-of-the-art models for aerial video saliency estimation.
• Achieved remarkable speeds of up to 28,738 FPS on GPU and 1,490.5 FPS on CPU.
• Effectively estimates dynamic saliency on aerial videos while addressing computational constraints.

Conclusions:

• The spatiotemporal knowledge distillation method offers an effective solution for efficient dynamic saliency estimation in aerial videos.
• The approach successfully balances performance and computational efficiency, making it suitable for edge devices.
• This work contributes to the practical application of advanced saliency estimation techniques in real-world scenarios with limited resources.