Interaction-Aware Shared Scene Synthesis for VR Telepresence.

Area of Science:

• Computer Science
• Human-Computer Interaction
• Virtual Reality

Background:

• Virtual reality telepresence enables real-time remote collaboration in shared virtual environments.
• Current methods struggle to synthesize virtual spaces that balance user interaction needs with physical constraints.

Purpose of the Study:

• To propose an interaction-aware shared virtual scene synthesis method using large language models (LLMs).
• To improve the accommodation of diverse user interaction requirements within synthesized virtual spaces.

Main Methods:

• Introduced Interaction Aware Templates (IATs) constructed using an LLM planner.
• Developed an IAT-based affordance field alignment for merging local user spaces.
• Proposed an LLM-based shared scene synthesis method guided by the merged affordance field.

Main Results:

• Achieved superior Affordance Consistency, 3D Intersection over Union, and Layout Suitability compared to existing methods.
• Demonstrated significant improvements in user-perceived interaction fitness and safety through user studies.

Conclusions:

• The proposed LLM-based method effectively synthesizes virtual scenes that support user interactions in telepresence.
• This approach enhances collaboration by creating safer and more suitable shared virtual environments.