Hierarchical Bayesian LSTM for Head Trajectory Prediction on Omnidirectional Images

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Machine Learning

Background:

• Head movement (HM) in omnidirectional images (ODIs) creates spatial-temporal head trajectories.
• Understanding these trajectories is crucial for modeling human attention on ODIs.

Purpose of the Study:

• To establish a large-scale dataset of head trajectories on ODIs.
• To identify key factors influencing head trajectories.
• To propose a novel approach for head trajectory prediction.

Main Methods:

• Developed a large-scale dataset of 21,600 head trajectories on 1,080 ODIs.
• Proposed HiBayes-LSTM, integrating hierarchical Bayesian inference with Long Short-Term Memory (LSTM) networks.
• Introduced Future Intention Estimation (FIE) for viewport transition prediction.
• Implemented Hierarchical Bayesian Inference (HBI) to model inter-subject uncertainty using a joint Gaussian distribution.

Main Results:

• Identified temporal dependency and subject-specific variance as key factors influencing head trajectories.
• HiBayes-LSTM successfully models inter-subject uncertainty, generating diverse viewport transitions.
• The approach significantly outperforms 9 state-of-the-art methods in head trajectory prediction.
• Successfully applied to predict saliency on ODIs.

Conclusions:

• HiBayes-LSTM provides a robust framework for predicting head trajectories in ODIs by accounting for temporal dynamics and individual differences.
• The method enhances the understanding of human attention in immersive environments.
• This work paves the way for more accurate saliency prediction models in ODIs.