Designing for Collaboration: Visualization to Enable Human-LLM Analytical Partnership

Area of Science:

• Data Visualization
• Human-Computer Interaction
• Artificial Intelligence

Background:

• Visualization artifacts traditionally support human collaboration and knowledge transfer in data analysis.
• The role of visualization in capturing knowledge during human-large language model (LLM) interaction remains underexplored.
• Current LLM workflows in analytics are often linear and text-based, hindering structured representation of the analytical process.

Purpose of the Study:

• To investigate the potential of visualization artifacts in human-LLM analytical workflows.
• To highlight the limitations of current LLM text-based approaches for tracking and structuring analysis.
• To advocate for dynamic visual representations to enhance human-LLM collaboration and knowledge externalization.

Main Methods:

• Exploration of current opportunities and limitations of LLMs in tracking, structuring, and visualizing analytic processes.
• Conceptual argument for the integration of dynamic visualization in human-LLM workflows.
• Proposal of a research agenda informed by LLM advancements.

Main Results:

• LLMs' linear text-based workflows limit the traceability and structure of analytical artifacts.
• Dynamic visual representations are proposed as critical for structuring evolving artifacts and provenance.
• Opportunities and limitations for using LLMs to visualize analytic processes are demonstrated.

Conclusions:

• Dynamic visualization is essential for effective human-LLM analytical interactions.
• Visualizing evolving artifacts and provenance can lead to more structured and transparent analytical processes.
• Further research is needed to leverage LLM capabilities for enhanced visualization in analytics.