LLMER: Crafting Interactive Extended Reality Worlds with JSON Data Generated by Large Language Models.

Area of Science:

• Computer Science
• Human-Computer Interaction
• Artificial Intelligence

Background:

• Integrating Large Language Models (LLMs) with Extended Reality (XR) enables natural language interaction for immersive environments.
• Current XR development faces challenges with data extraction complexity, leading to high costs and generation errors.
• Existing script generation methods often produce flawed code, causing application instability and poor user experience.

Purpose of the Study:

• To introduce LLMER, a novel framework for generating interactive XR worlds using LLM-generated JSON data.
• To address the limitations of current XR development approaches, particularly those relying on script generation.
• To enhance the efficiency and reduce errors in creating complex XR environments.

Main Methods:

• LLMER translates natural language inputs into structured JSON data, bypassing direct code generation.
• A multi-stage strategy selectively provides essential contextual information tailored to user requests.
• The framework incorporates multiple modules designed for diverse XR application requirements.

Main Results:

• LLMER demonstrated an over 80% reduction in consumed tokens compared to state-of-the-art methods.
• Task completion time was reduced by approximately 60% in preliminary user studies.
• The JSON-based approach significantly minimizes application crashes and processing latency.

Conclusions:

• LLMER offers a more robust and efficient method for developing interactive XR experiences.
• The framework effectively reduces errors and improves performance in LLM-powered XR content creation.
• User feedback highlights the potential for further optimization and broader application of the LLMER system.