Task Breakpoint Generation using Origin-Centric Graph in Virtual Reality Recordings for Adaptive Playback

Area of Science:

• Computer Science
• Human-Computer Interaction
• Virtual Reality

Background:

• Adaptive tutorials and authoring tools are increasingly important with the rise of Augmented Reality (AR)/Virtual Reality (VR) head-mounted displays (HMDs).
• Existing task segmentation methods often rely on manual annotation or are limited to 2D video, hindering their application in 3D VR environments.
• There is a need for automated methods to segment complex activities in VR for personalized learning experiences.

Purpose of the Study:

• To propose and validate a novel method for automatic task breakpoint generation in goal-oriented activity recordings within VR.
• To enable adaptive playback of VR content by segmenting activities into meaningful task units.
• To overcome limitations of existing methods in 3D VR contexts.

Main Methods:

• Recording assembly scenarios with defined task boundaries using a structured spatio-temporal scene graph (STSG).
• Employing an Origin-Centric Graph (OCG) to track object changes and group formations for automatic breakpoint detection.
• Conducting a user study to collect perceived task breakpoints for ground truth (GT) comparison.

Main Results:

• The proposed OCG-based method demonstrated high agreement with user-perceived ground truth breakpoints.
• Algorithm-detected breakpoints were confirmed to be accurate, effectively supporting adaptive VR playback.
• The method successfully segmented goal-oriented activities into distinct task units.

Conclusions:

• The OCG-based task segmentation method provides a robust foundation for dynamic VR playback adjustment based on user progress.
• This approach facilitates personalized VR learning by adapting content delivery.
• The method has potential for broader applications in automatic timeline segmentation for various VR recordings.