Revisiting Video Saliency Prediction in the Deep Learning Era.

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Cognitive Science

Background:

• Predicting visual saliency in static scenes is well-researched, but modeling attention in dynamic scenes remains a challenge.
• Existing methods lack comprehensive benchmarks and efficient models for dynamic visual attention.

Purpose of the Study:

• Introduce the Dynamic Human Fixation 1K (DHF1K) dataset for video saliency research.
• Propose the Attentive CNN-LSTM Network (ACLNet) for accurate and efficient video saliency prediction.
• Evaluate state-of-the-art models and the proposed ACLNet on multiple datasets.

Main Methods:

• Developed DHF1K: 1K high-quality video sequences with human eye-tracking data.
• Proposed ACLNet: A CNN-LSTM model with a supervised attention mechanism integrating static saliency.
• Conducted extensive evaluations on DHF1K, Hollywood-2, and UCF Sports datasets.

Main Results:

• ACLNet demonstrates superior performance in predicting video saliency compared to other models.
• ACLNet achieves fast processing speeds (40 fps on a single GPU).
• Extensive analysis of saliency models and cross-dataset generalization is provided.

Conclusions:

• The DHF1K dataset and ACLNet model significantly advance video saliency prediction.
• ACLNet's attention mechanism enhances temporal saliency learning and model efficiency.
• The findings offer valuable insights for understanding and modeling dynamic visual attention.