DracoGPT: Extracting Visualization Design Preferences from Large Language Models

Area of Science:

• Computer Science
• Human-Computer Interaction
• Data Visualization

Background:

• Large Language Models (LLMs) trained on extensive data may encode visualization design knowledge.
• However, their learned preferences might not align with established best practices, leading to unreliable recommendations.

Purpose of the Study:

• To develop a method (DracoGPT) for extracting, modeling, and assessing visualization design preferences learned by LLMs.
• To compare LLM-derived preferences against established visualization design guidelines.

Main Methods:

• Developed two pipelines, DracoGPT-Rank and DracoGPT-Recommend, to model LLM preferences for ranking and recommending visual encoding specifications.
• Utilized Draco, a shared knowledge base, to represent and analyze LLM preferences against empirical research guidelines.

Main Results:

• DracoGPT accurately models the visualization design preferences of various LLMs.
• Both DracoGPT-Rank and DracoGPT-Recommend showed moderate agreement but substantially diverged from human-based experimental guidelines.

Conclusions:

• LLMs exhibit distinct visualization design preferences that can be modeled and analyzed.
• Current LLM preferences in visualization design deviate significantly from human-validated best practices, highlighting a need for refinement.