Separating the Wheat from the Chaff: Comparative Visual Cues for Transparent Diagnostics of Competing Models

Area of Science:

• Data and physical sciences
• Climate science

Background:

• Evaluating competing analytical or physics-based models is a significant challenge for experts.
• Reconciling conflicting criteria to assess model outcome quality requires robust diagnostic methods.

Purpose of the Study:

• To understand the problem and solution space of model diagnostics through a design study with climate scientists.
• To develop methods for experts to reliably diagnose which model outcomes best predict real-world phenomena.

Main Methods:

• Characterizing expert model diagnostic goals and mapping them to multi-way visual comparison tasks.
• Designing a space of comparative visual cues to assess model disagreement and output stability.
• Developing and evaluating MyriadCues, an interactive visualization interface.

Main Results:

• A problem and task characterization for model diagnostics.
• A design space of comparative visual cues for model comparison.
• The MyriadCues interface, enabling transparent model diagnostic mechanisms.

Conclusions:

• MyriadCues facilitates a deeper understanding of model performance by leveraging comparative visual cues.
• The developed visualization interface enhances the transparency and effectiveness of model diagnostic processes.
• Expert feedback validates MyriadCues' ability to support the exploration of alternative hypotheses and model insights.