Related Experiment Video
Updated: Feb 5, 2026
06:45
Task Interruption and Resumption Paradigm for Testing the Activation and Pursuit of an Abstract Thinking Goal
Published on: April 18, 2017
6.6K
In Pursuit of Error: A Survey of Uncertainty Visualization Evaluation
IEEE Transactions on Visualization and Computer Graphics
|September 13, 2018
Summary
Evaluating uncertainty visualizations is complex. This study introduces a taxonomy to guide evaluations, revealing current methods often overlook human judgment complexities, suggesting a need for better decision quality assessment in visualization research.
Area of Science:
- Information Visualization
- Human-Computer Interaction
- Cognitive Psychology
Background:
- Effective reasoning with visualized data requires understanding uncertainty.
- Evaluating uncertainty visualizations is challenging due to interpretation difficulties and defining correct behavior.
- Existing general-purpose frameworks inadequately address the unique complexities of assessing judgments under uncertainty.
Purpose of the Study:
- To present a novel taxonomy for characterizing design decisions in uncertainty visualization evaluations.
- To provide guidance for researchers and practitioners evaluating uncertainty visualizations.
- To identify and address the mismatch between uncertainty conceptualization in visualization and other fields.
Main Methods:
- Developed a taxonomy with six decision levels: behavioral targets, expected effects, evaluation goals, measures, elicitation techniques, and analysis approaches.
- Analyzed 86 user studies of uncertainty visualizations using the proposed taxonomy.
- Reflected on common themes in evaluation practices, including interpretation, semantics, confidence reporting, and performance metrics.
Main Results:
- Existing evaluation practices, especially in visualization research, predominantly use performance and satisfaction measures.
- These measures often assume predictable, statistically-driven judgment behavior, contrasting with research on human decision-making.
- A bias exists towards evaluating performance as accuracy rather than decision quality.
Conclusions:
- The proposed taxonomy offers a structured approach to designing uncertainty visualization evaluations.
- Current evaluation methods may not fully capture the nuances of human judgment with uncertainty.
- Recommendations are provided to align visualization evaluation practices with broader research on human judgment and decision-making, emphasizing decision quality over mere accuracy.
Related Concept Videos
01:19Errors and Mistakes in Surveying
714
Errors and mistakes in surveying refer to inaccuracies in measurements and data recording. The errors are deviations from the actual value caused by human sensory limitations, equipment flaws, or environmental effects. These errors are typically unintentional and can result from the inherent imperfections in the instruments used, atmospheric conditions, or the observer’s inability to perceive exact measurements. On the other hand, mistakes are caused by the surveyor's lack of...
714
00:59Propagation of Uncertainty from Random Error
2.0K
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
2.0K
01:10Propagation of Uncertainty from Systematic Error
1.5K
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
1.5K
02:16Surveys
16.9K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
16.9K
04:08The Uncertainty Principle
32.0K
Werner Heisenberg considered the limits of how accurately one can measure properties of an electron or other microscopic particles. He determined that there is a fundamental limit to how accurately one can measure both a particle’s position and its momentum simultaneously. The more accurate the measurement of the momentum of a particle is known, the less accurate the position at that time is known and vice versa. This is what is now called the Heisenberg uncertainty principle. He...
32.0K
02:46Uncertainty in Measurement: Reading Instruments
52.9K
Counting is the type of measurement that is free from uncertainty, provided the number of objects being counted does not change during the process. Such measurements result in exact numbers. By counting the eggs in a carton, for instance, one can determine exactly how many eggs are there in the carton. Similarly, the numbers of defined quantities are also exact. For example, 1 foot is exactly 12 inches, 1 inch is exactly 2.54 centimeters, and 1 gram is exactly 0.001 kilograms. Quantities...
52.9K