Related Experiment Video
Updated: Sep 10, 2025
11:34
High-resolution, High-speed, Three-dimensional Video Imaging with Digital Fringe Projection Techniques
Published on: December 3, 2013
15.7K
Viewport prediction with cross modal multiscale transformer for 360° video streaming
Yangsheng Tian1, Yi Zhong2, Yi Han3
1School Of Information Engineering, Wuhan University of Technology, LuoShi Road, Wuhan, 430070, Hubei, China. 369487529@qq.com.
Scientific Reports
|August 19, 2025
Summary
This study introduces a new model for predicting user views in 360° video streaming. The Cross Modal Multiscale Transformer (CMMST) improves prediction accuracy by considering user movement and visual importance.
Area of Science:
- Computer Science
- Multimedia Systems
- Artificial Intelligence
Background:
- Efficient 360° video streaming faces challenges with high bandwidth demands and unpredictable user viewports.
- Current methods often overlook inter-modal dependencies and personalized user preferences, leading to suboptimal prediction accuracy.
Purpose of the Study:
- To develop a novel viewport prediction model for 360° video streaming.
- To enhance prediction accuracy and efficiency by integrating diverse features and user-specific data.
Main Methods:
- A Cross Modal Multiscale Transformer (CMMST) model was developed.
- The model integrates user trajectory and video saliency features across multiple scales.
- Cross-modal attention mechanisms were employed to capture user preferences and viewing patterns.
Main Results:
- The CMMST model demonstrated superior performance compared to baseline methods.
- High prediction precision was maintained even with longer prediction intervals.
- The model effectively captures intricate user preferences and viewing behaviors.
Conclusions:
- The proposed CMMST model offers a promising solution for adaptive 360° video streaming.
- This approach can significantly improve user experience in virtual reality and other immersive platforms.
- The integration of multimodal features and attention mechanisms is key to accurate viewport prediction.
Related Concept Videos
01:25Relative Motion Analysis using Rotating Axes
531
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
531
01:16Vector Transformation in Rotating Coordinate Systems
1.8K
Consider a vector rotating about an axis with an angular velocity, such that its tip sweeps a circular path.
1.8K
01:25Cross Product
330
The cross product is a fundamental concept in vector algebra that is a vector operation on two different vectors to obtain a third vector. Unlike the scalar product, the cross product results in a vector quantity perpendicular to both the original vectors.
The magnitude of the cross product is obtained by multiplying the magnitude of both the vectors and the sine of the angle between them. This means that a larger angle between the vectors will lead to a greater magnitude of the cross product.
The magnitude of the cross product is obtained by multiplying the magnitude of both the vectors and the sine of the angle between them. This means that a larger angle between the vectors will lead to a greater magnitude of the cross product.
330
01:27Uniform Depth Channel Flow
144
Uniform depth channel flow keeps fluid depth consistent along channels such as irrigation canals. In natural channels, such as rivers, approximate uniform flow is often assumed. This condition occurs when the channel’s bottom slope matches the energy slope, balancing potential energy lost from gravity with head loss due to shear stress. This balance prevents depth changes along the channel length, resulting in a steady, uniform flow.Uniform flow in open channels with a constant cross-section...
144
01:15Depth Perception and Spatial Vision
900
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
900
01:26Scaling
315
In designing and analyzing filters, resonant circuits, or circuit analysis at large, working with standard element values like 1 ohm, 1 henry, or 1 farad can be convenient before scaling these values to more realistic figures. This approach is widely utilized by not employing realistic element values in numerous examples and problems; it simplifies mastering circuit analysis through convenient component values. The complexity of calculations is thereby reduced, with the understanding that...
315