Qualitative Study for LLM-assisted Design Study Process: Strategies, Challenges, and Roles.

Area of Science:

• Human-Computer Interaction
• Information Visualization
• Artificial Intelligence

Background:

• Design studies are crucial for developing effective visualization solutions for real-world problems.
• Large language models (LLMs) present emerging opportunities to enhance the design study process through capabilities like creative problem-solving and data analysis.
• A systematic understanding of how LLMs can assist researchers in visualization-specific design studies is currently lacking.

Purpose of the Study:

• To systematically investigate the role and impact of large language models (LLMs) in visualization design studies.
• To identify effective strategies, challenges, and best practices for utilizing LLMs in the design study process within visualization research.
• To develop a framework for leveraging LLMs to facilitate and enhance visualization design studies.

Main Methods:

• A multi-stage qualitative study involving 30 design study researchers with diverse backgrounds and expertise levels.
• In-depth interviews and carefully designed questionnaires to gather insights on LLM utilization strategies, encountered challenges, and mitigation practices.
• Compilation of the potential roles LLMs can assume across various stages of the design study process.

Main Results:

• Identified specific strategies and practical approaches for integrating LLMs into different phases of visualization design studies.
• Documented common challenges faced by researchers when using LLMs and outlined effective practices for overcoming these obstacles.
• Cataloged the diverse roles LLMs can fulfill, from ideation and data handling to analysis and refinement, within the design study workflow.

Conclusions:

• LLMs can significantly enhance the design study process in visualization research by offering novel capabilities and support.
• Practical implications are provided for visualization practitioners on how to effectively leverage LLMs.
• A foundational framework is proposed for the systematic integration of LLMs to advance the field of visualization design studies.