How Visually Literate Are Large Language Models? Reflections on Recent Advances and Future Directions

Area of Science:

• Computer Science
• Data Visualization
• Artificial Intelligence

Background:

• Large language models (LLMs) are increasingly utilized for visualization tasks.
• These models demonstrate potential for "visual literacy" in both generation and interpretation.

Purpose of the Study:

• To review the current state of LLM visualization literacy.
• To explore future directions for LLM applications in data visualization.

Main Methods:

• Analysis of LLM capabilities in visualization generation (e.g., natural language interfaces for authoring).
• Evaluation of LLM performance in visualization interpretation (e.g., answering questions, synthesizing information, detecting misleading designs).

Main Results:

• LLMs show potential in generating visualizations via natural language but have usability issues.
• Models can perform basic interpretation tasks but struggle with complex analytics and human-like understanding.
• Discrepancies exist between LLM and human interpretation of visualizations.

Conclusions:

• LLMs offer advancements in visualization generation and interpretation.
• Further research is needed to address limitations in analytical tasks and human-aligned understanding.
• Future work should focus on improving LLM visualization literacy for more robust applications.