Related Experiment Video
Updated: Jun 2, 2026

Three-Dimensional Mapping of the Rotation of Interactive Virtual Objects with Eye-Tracking Data
Published on: October 18, 2024
Large Language Model-Based Interactive Code Generation for Developing a 3D Eye Movement Schematic
Ichiro Hamasaki1, Keiko Kunimi2, Kiyo Shibata1
1Ophthalmology, Lino Eye Clinic, Okayama, JPN.
Abstract:
Introduction Understanding the three-dimensional (3D) geometric relationship between extraocular muscles and the globe is essential for strabismus management. Conventional educational tools are static, and existing 3D biomechanical software requires highly specialized skills, making routine clinical use difficult. Furthermore, image-generating artificial intelligence (AI) frequently produces anatomically incorrect outputs (hallucinations). This study aimed to develop a structurally coherent, interactive 3D eye movement schematic as a proof-of-concept, using the coding capabilities of a large language model (LLM). Methods We used an LLM to generate web-based 3D schematic code (HTML and JavaScript/Three.js) exclusively through natural language dialogue. To prevent anatomical errors, we explicitly defined anatomical parameters based on standard literature (e.g., 12-mm scleral radius) and employed mathematical constraints, including quaternions for rotation and spherical linear interpolation for muscle paths, within the prompts. The generated code was rendered in a web browser, and an iterative process of prompt refinement and debugging was conducted until two board-certified ophthalmologists confirmed the schematic's structural validity. Results A functional, interactive 3D eye movement schematic was successfully developed. In our 10-trial evaluation, generating an acceptable schematic required an average of 7.4 prompt inputs per session. While complex spatial instructions had a lower success rate (40%) due to AI hallucinations, iterative prompt repetition and specific local debugging instructions resolved these issues. The final schematic provided a structurally coherent representation of the globe, the four rectus muscles, and the annulus of Zinn. It featured a slider interface enabling real-time, kinematic visualizations of eye rotations, muscle deformations, and optic nerve bending without structural failure. Conclusions Translating anatomical descriptions into mathematical spatial logic via LLMs enables the creation of structurally sound 3D medical schematics. This logical spatial construction approach democratizes the development of interactive educational tools. It allows healthcare providers without programming expertise to intuitively generate customizable 3D educational materials for patient consultations and foundational medical education through natural language dialogue.
