Video Saliency Prediction using Spatiotemporal Residual Attentive Networks

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Predicting human eye movements in videos is crucial for understanding visual attention.
• Existing models often struggle with effective spatiotemporal feature integration and multi-scale saliency learning.

Purpose of the Study:

• To propose a novel residual attentive learning network architecture for dynamic eye-fixation map prediction.
• To address challenges in spatiotemporal feature integration and multi-scale saliency learning.

Main Methods:

• A novel residual attentive learning network architecture is proposed.
• Dense residual cross-connections tightly couple appearance and motion streams for unified spatiotemporal learning.
• A composite attention mechanism is introduced for multi-scale local and global attention learning.
• A lightweight convolutional Gated Recurrent Unit (convGRU) models long-term temporal characteristics.

Main Results:

• The proposed model demonstrates superior performance in predicting dynamic eye-fixation maps.
• Experiments on four benchmark datasets validate the effectiveness of individual network components.
• The approach outperforms existing video saliency models.

Conclusions:

• The novel architecture effectively integrates spatiotemporal features and learns multi-scale saliency.
• The model offers a powerful solution for video saliency prediction tasks.
• The approach is robust and effective, even with limited training data.