Augmented saliency model using automatic 3D head pose detection and learned gaze following in natural scenes

Area of Science:

• Computer Vision
• Cognitive Science
• Human-Computer Interaction

Background:

• Actor gaze direction influences observer eye movements during free-viewing.
• Existing computational models lack integration of bottom-up saliency with head pose and gaze direction for fixation prediction.

Purpose of the Study:

• To develop a computational model combining bottom-up saliency, head pose, and gaze direction for predicting observer fixations.
• To evaluate the model's performance with both oracle and automated head pose data.

Main Methods:

• Learned probability maps for gaze-following and head fixations based on head size and pose.
• Developed a Markov chain model integrating saliency, head regions, and gaze following.
• Assessed model performance using oracle head pose and subsequently with computer vision-detected head pose.

Main Results:

• The Dynamic Weighting of Cues (DWOC) model significantly outperformed individual components in predicting observer fixations.
• The combined model showed significant improvement even with imperfect, automated head pose detections.
• The model provides insights into eye movement mechanisms originating from head regions.

Conclusions:

• Integrating bottom-up saliency with head pose and gaze direction is effective for predicting visual attention.
• The DWOC model offers a robust framework for understanding and engineering visual attention mechanisms.
• This research advances saliency models for both scientific and engineering applications.