Graph Learning Based Head Movement Prediction for Interactive 360 Video Streaming.

Area of Science:

• Computer Science
• Multimedia Systems
• Machine Learning

Background:

• Ultra-high definition (UHD) 360 videos are bandwidth-intensive, posing challenges for streaming over limited networks.
• Interactive extraction of spatial sub-regions based on viewer's field-of-view (FoV) is a common BW-efficient streaming method.
• Accurate head movement prediction is vital to mitigate latency issues caused by round-trip-time (RTT) delays in head-mounted display (HMD) streaming.

Purpose of the Study:

• To develop an accurate head movement prediction model for optimizing UHD 360 video streaming.
• To improve bandwidth efficiency and rate-distortion performance in interactive video streaming systems.
• To integrate viewer behavior and visual saliency into a predictive model for enhanced streaming.

Main Methods:

• Head movement prediction is framed as a sparse directed graph learning problem.
• A view transition Markov model is constructed using viewer traces, saliency maps, and a human head model.
• A constrained maximum a posteriori (MAP) problem is solved using a hybrid iterative reweighted least square (IRLS) and Frank-Wolfe (FW) optimization strategy.

Main Results:

• The proposed head movement prediction scheme significantly outperformed existing methods.
• The optimized tile-based streaming system demonstrated superior rate-distortion performance compared to competitors.
• The use of viewer head movement traces, saliency maps, and a head model improved prediction accuracy.

Conclusions:

• The developed graph learning approach provides a robust method for head movement prediction in 360 video streaming.
• Optimizing tile-based streaming with accurate prediction enhances user experience in bandwidth-constrained environments.
• This research contributes to more efficient delivery of immersive media content.