Decision criteria do not shift: commentary on Mueller and Weidemann (2008)

Area of Science:

• Cognitive psychology
• Decision-making research
• Psychophysics

Background:

• Receiver operating characteristic (ROC) curves in signal detection theory (SDT) are typically assumed to be invariant to base rates and payoffs.
• However, empirical data, particularly from rating tasks, show deviations from this assumption, challenging the standard SDT model's notion of a stable decision criterion.

Discussion:

• This work critically examines the decision-noise hypothesis proposed by Mueller and Weidemann (2008) as an explanation for these deviations.
• The critique highlights that the hypothesis does not adequately account for the fundamental inconsistencies observed in rating data and other detection paradigms.
• Furthermore, the decision-noise hypothesis's predictions regarding the effect of controlling for response variability were empirically contradicted.

Key Insights:

• The observed inconsistencies in ROC curve shapes challenge the core assumptions of signal detection theory, particularly the stability of decision criteria.
• The decision-noise hypothesis, while attempting to explain these anomalies, is shown to be insufficient and its predictions are not supported by experimental evidence.
• Violations of SDT assumptions are present not only in rating tasks but also in yes-no tasks when response time is analyzed.

Outlook:

• Further research is needed to refine or replace existing models of decision-making under uncertainty.
• Investigating alternative theoretical frameworks that can better accommodate the observed variability in decision criteria is crucial.
• Future studies should focus on developing models that explain both rating and yes-no task data consistently.