How is Gaze Influenced by Image Transformations? Dataset and Model.

Area of Science:

• Computer Vision
• Cognitive Science
• Machine Learning

Background:

• Deep saliency models require large datasets, but collecting eye-movement data is time-consuming and expensive.
• Existing saliency research often uses high-quality, unaltered images, which do not reflect real-world scenarios.
• Image transformations are common in real-world captured images, presenting an opportunity for data augmentation.

Purpose of the Study:

• To investigate the use of image transformations for augmenting saliency datasets.
• To develop and evaluate a novel deep saliency model robust to image transformations.
• To establish a new benchmark for saliency model robustness.

Main Methods:

• Created a new saliency dataset with human eye-movement data over 1900 transformed images.
• Analyzed human gaze patterns on original versus transformed images.
• Trained deep saliency models using data augmentation transformations (DAT) and introduced the GazeGAN model.
• Utilized a modified U-Net with a novel center-surround connection (CSC) module in GazeGAN.

Main Results:

• Label-preserving DATs significantly improved saliency prediction performance.
• DATs that altered human gaze patterns degraded model performance.
• GazeGAN demonstrated state-of-the-art performance across multiple datasets.
• A comprehensive comparison of 22 saliency models on transformed scenes was provided.

Conclusions:

• Label-preserving image transformations offer a viable method to enlarge existing saliency datasets.
• The proposed GazeGAN model exhibits superior robustness and performance on transformed images.
• The study provides valuable insights and a robustness benchmark for the saliency modeling community.