Visual Reasoning Strategies for Effect Size Judgments and Decisions.

Area of Science:

• Data visualization
• Human-computer interaction
• Cognitive psychology

Background:

• Uncertainty visualizations often prioritize point estimates, potentially leading users to misinterpret visual distance as effect size.
• This can cause users to overlook crucial uncertainty information when making magnitude estimates or decisions.

Purpose of the Study:

• To investigate the impact of adding means to uncertainty visualizations on magnitude estimation and decision-making.
• To compare the effectiveness of different uncertainty visualization designs (95% containment intervals, hypothetical outcome plots, densities, quantile dotplots) with and without means.

Main Methods:

• A mixed-design experiment was conducted on Mechanical Turk.
• Eight uncertainty visualization designs were tested, varying in representation (intervals, plots, densities, dotplots) and inclusion of means.
• Qualitative analysis of user strategy descriptions was performed.

Main Results:

• Adding means to uncertainty visualizations had small biasing effects on magnitude estimation and decision-making, suggesting users discounted uncertainty.
• Visualization designs that minimized biased effect size estimation did not necessarily lead to the best decision-making.
• Many users switched strategies and did not consistently employ optimal approaches.

Conclusions:

• Theoretical optimal uncertainty visualization designs may not be most effective in practice due to user heuristics and satisficing.
• Understanding user strategy selection is crucial for improving the practical effectiveness of uncertainty visualizations.
• Further research should model user strategies to better understand visualization effectiveness.