Revealing Perceptual Proxies with Adversarial Examples.

Area of Science:

• Human-Computer Interaction
• Cognitive Psychology
• Data Visualization

Background:

• Data visualizations translate numerical data into visual representations.
• The human visual system interprets these visuals using perceptual proxies (heuristic shortcuts) rather than direct computation.
• Understanding these proxies is crucial for designing effective visualizations.

Purpose of the Study:

• To measure the power of candidate perceptual proxies in explaining human performance when comparing data series in bar charts.
• To identify which visual shortcuts viewers employ when interpreting data visualizations.
• To investigate individual differences in proxy usage.

Main Methods:

• Conducted crowdsourced experiments comparing pairs of data series in bar charts.
• Generated adversarial datasets pitting correct answers against visually misleading series based on candidate proxies.
• Employed Bayesian logistic regression and Bayesian mixed-effects linear models to analyze viewer responses.
• Utilized iterative crowdsourcing for black-box optimization to construct adversarial datasets.

Main Results:

• Quantified the influence of specific perceptual proxies on viewer interpretation of bar chart data.
• Determined the extent to which adversarial proxies could lead to incorrect judgments.
• Provided evidence for potential individual variations in the use of perceptual proxies.

Conclusions:

• Perceptual proxies significantly impact data interpretation in bar charts.
• Identifying and understanding these proxies can inform the design of less ambiguous and more effective data visualizations.
• Further research can leverage these findings to develop optimized visualization strategies.