Finding useful questions: on Bayesian diagnosticity, probability, impact, and information gain

Area of Science:

• Cognitive Science
• Decision Making
• Information Theory

Background:

• Several norms exist for assessing question usefulness in decision-making.
• These include Bayesian diagnosticity, information gain, Kullback-Liebler distance, probability gain, and impact.
• Previous models failed to distinguish between these norms in human behavior.

Purpose of the Study:

• To evaluate the normative and empirical validity of Bayesian diagnosticity compared to other measures.
• To identify situations where different usefulness norms yield contradictory predictions.

Main Methods:

• Computational optimization to find conflicting scenarios between norms.
• Analysis of existing experimental data from categorization, covariation, medical diagnosis, and selection tasks.
• Conducting a new experiment to test predictions.

Main Results:

• Information gain, probability gain, and impact were found to contradict Bayesian diagnosticity in computational models.
• Existing experimental data could not differentiate between the norms.
• A new experiment's results contradicted Bayesian diagnosticity's predictions.

Conclusions:

• Bayesian diagnosticity is normatively inferior and empirically unjustified as a model of human judgment.
• Alternative measures like information gain offer better explanations for assessing question usefulness.
• The study highlights flaws in relying solely on Bayesian diagnosticity for understanding decision-making.