Saliency Prediction in the Deep Learning Era: Successes and Limitations

Area of Science:

• Computer Vision
• Artificial Intelligence
• Cognitive Science

Background:

• Deep learning and large-scale datasets have significantly improved visual saliency models.
• Despite progress, current models do not achieve human-level accuracy in predicting visual attention.

Purpose of the Study:

• To review and compare recent deep learning-based visual saliency models.
• To identify factors contributing to the performance gap between models and human visual attention.
• To discuss challenges and future directions for developing next-generation saliency models.

Main Methods:

• A comprehensive review of existing image and video saliency models.
• Comparative analysis of models using two image benchmarks and two large-scale video datasets.
• Exploration of cognitive studies on attention and their relevance to computational models.

Main Results:

• Current deep saliency models show significant improvements but still lag behind human performance.
• Identified specific failure modes of current models and potential remedies.
• Highlighted the importance of cognitive insights for enhancing saliency prediction.

Conclusions:

• Bridging the gap requires addressing model limitations informed by human attention research.
• Fair model comparison methodologies and new datasets are crucial for progress.
• Emerging applications of visual saliency models are expanding.