Toward Mixed Method Evaluations of Scientific Visualizations and Design Process as an Evaluation Tool

Area of Science:

• Scientific Visualization
• Human-Computer Interaction
• Information Visualization

Background:

• Current evaluation strategies for scientific visualizations have limitations.
• Traditional user studies provide valuable but sometimes insufficient feedback.
• Design fields offer established methodologies for evaluation.

Purpose of the Study:

• To advocate for a mixed methods evaluation strategy for scientific visualizations.
• To propose the integration of design processes as a formal evaluation mode.
• To highlight the benefits of design-based evaluation for visualization systems.

Main Methods:

• Discussing successes and limitations of current evaluation strategies.
• Describing recent evaluations of scientific visualization interfaces and computer graphics strategies.
• Employing design methods such as sketching and critique alongside traditional user studies.

Main Results:

• Design-based evaluation methods provide more detailed feedback than traditional user studies.
• Sketching, critique, and ideation offer invaluable insights into visualization systems.
• Recent evaluations demonstrate the effectiveness of complementary design approaches.

Conclusions:

• A mixed methods strategy combining traditional and design-based evaluations is recommended.
• Embracing design processes can significantly enhance the evaluation of scientific visualizations.
• Design-based evaluation methodologies hold potential for future formative and summative evaluations.