Semantic-Driven Generation of Hyperlapse from 360 Degree Video.

Area of Science:

• Computer Vision
• Digital Media Processing
• Human-Computer Interaction

Background:

• Panoramic (360-degree) videos offer immersive experiences but can be challenging to navigate.
• Hyperlapse videos condense time and space, providing dynamic overviews.
• Current methods for generating hyperlapses from 360-degree content may not fully leverage visual importance or user preferences.

Purpose of the Study:

• To develop an automated system for converting 360-degree videos into normal field-of-view (NFOV) hyperlapses.
• To enhance the viewing experience by employing visual saliency and semantic understanding for non-uniform spatial and temporal sampling.
• To enable user customization of hyperlapses by allowing selection of objects of interest.

Main Methods:

• Video stabilization of input 360-degree footage.
• Computation of regions of interest and saliency scores.
• Optimization of saliency and motion smoothness for initial hyperlapse generation.
• Saliency-aware frame selection and adaptive 2D video stabilization for final output.

Main Results:

• A novel system capable of generating NFOV hyperlapses from 360-degree videos.
• Demonstrated effectiveness across diverse scenes.
• User study results indicate competitive or superior performance compared to state-of-the-art methods.

Conclusions:

• The proposed system effectively converts 360-degree videos into perceptually optimized NFOV hyperlapses.
• Exploiting visual saliency and semantics significantly improves hyperlapse quality and user engagement.
• The system offers a valuable tool for content creators and viewers seeking enhanced video experiences.