Motion parallax for 360° RGBD video.

Area of Science:

• Computer Vision
• Virtual Reality
• Computer Graphics

Background:

• Current 360° video playback in VR lacks responsiveness to head movement, diminishing immersion and causing discomfort.
• Existing methods using depth maps for parallax introduce visual artifacts like distortions and holes at depth discontinuities.

Purpose of the Study:

• To develop a novel method for incorporating parallax and real-time playback of 360° videos in virtual reality (VR) headsets.
• To enhance the immersive experience and mitigate motion sickness associated with VR video viewing.

Main Methods:

• Improved depth map generation to create cleaner, more natural silhouettes.
• Implemented a three-layer scene representation (foreground, two background layers) to manage disocclusions.
• Utilized multi-frame information propagation and inpainting techniques for background layers.

Main Results:

• The method successfully generates more natural silhouettes, overcoming limitations of naive image-based rendering.
• The system effectively handles disocclusions, providing a seamless viewing experience.
• User studies confirmed increased immersion and reduced discomfort compared to non-parallax playback.

Conclusions:

• The proposed method significantly enhances the realism and comfort of 360° video playback in VR.
• It offers a robust solution compatible with various capture devices and adaptable even without initial depth data.
• This advancement contributes to a more compelling and accessible virtual reality viewing experience.