MineVRA: Exploring the Role of Generative AI-Driven Content Development in XR Environments through a Context-Aware...

Area of Science:

• Intersection of Artificial Intelligence (AI), Computer Vision (CV), Computer Graphics (CG), and Extended Reality (XR).
• Human-Computer Interaction (HCI) within immersive digital environments.

Background:

• Personalized 3D asset creation for XR is challenging, often relying on manual modeling.
• Generative AI (GenAI) presents a potential solution for automated, context-aware content generation.
• Existing 3D asset libraries may not fully meet the demand for personalized, adaptive content in XR.

Purpose of the Study:

• Introduce MineVRA, a novel Human-In-The-Loop (HITL) XR framework using GenAI for 3D asset creation.
• Evaluate the effectiveness and user satisfaction of GenAI-generated 3D assets compared to traditional sources.
• Explore the design implications of GenAI for human-centered XR development.

Main Methods:

• Developed MineVRA, a multimodal GenAI framework for adaptive 3D content generation in XR.
• Integrated GenAI with Human-In-The-Loop (HITL) processes for coherent asset creation.
• Conducted a comparative user study assessing GenAI-generated assets against Sketchfab assets in immersive contexts.

Main Results:

• GenAI-generated 3D objects demonstrated comparable performance and user satisfaction to traditionally sourced assets.
• MineVRA framework successfully facilitated coherent and adaptive 3D content generation.
• User study results indicate GenAI can significantly augment traditional 3D asset libraries.

Conclusions:

• GenAI, integrated via frameworks like MineVRA, offers a powerful tool for personalized 3D asset creation in XR.
• The findings support the use of GenAI to enhance the development of human-centered and adaptive immersive environments.
• This approach has valuable design implications for the future of XR content creation and user experience.