MoGraphGPT: Creating Interactive Scenes Using Modular LLM and Graphical Control.

Area of Science:

• Computer Graphics
• Human-Computer Interaction
• Artificial Intelligence

Background:

• Creating interactive 2D scenes typically requires complex programming.
• Existing large language models (LLMs) generate code but often produce errors and lack precise graphical control for multi-element interactions.

Purpose of the Study:

• To develop a system that simplifies the creation of 2D interactive scenes with multiple elements.
• To address limitations of current LLM-based code generation, specifically error-proneness and lack of graphical control.

Main Methods:

• Implemented a context-aware modularization technique using separate LLM modules for individual elements and a central module for interactions.
• Designed MoGraphGPT, a graphical user interface integrating modular LLMs with enhanced graphical controls like sliders.
• Utilized a top-down structure for managing interactions and ensuring clear update logic.

Main Results:

• MoGraphGPT significantly improves easiness, controllability, and performance compared to Cursor Composer in creating 2D interactive scenes.
• An ablation study confirmed the effectiveness of the modularization technique.
• An open-ended study demonstrated the system's usability and expressiveness for coding-free scene generation.

Conclusions:

• MoGraphGPT offers an effective solution for coding-free creation of complex 2D interactive scenes.
• The modular LLM approach combined with graphical control enhances user experience and output quality.