Signal detection models as contextual bandits.

Area of Science:

• Cognitive Science
• Decision Science
• Machine Learning

Background:

• Signal detection theory (SDT) traditionally assumes immediate adoption of optimal decision thresholds.
• This assumption is often violated as optimal responses frequently require learning.
• Uncertainty in decision-making necessitates adaptive strategies beyond static thresholds.

Purpose of the Study:

• To reframe classical signal detection models within a contextual multi-armed bandit (CMAB) framework.
• To investigate how decision-makers learn to infer cue-probability relationships under uncertainty.
• To compare CMAB heuristics with human behavior in signal detection tasks.

Main Methods:

• Recasting normal-normal and power-law signal detection models as CMABs.
• Developing and analyzing CMAB heuristics for balancing exploration and exploitation.
• Empirically testing CMAB predictions against human volunteer decisions in continuous cue signal detection tasks.

Main Results:

• A standard SDT model failed to predict human behavior accurately.
• The Softmax rule from CMAB best explained volunteer decisions, utilizing a logistic function of expected payoffs.
• A simple midpoint algorithm also showed predictive power under specific conditions.

Conclusions:

• Contextual multi-armed bandits provide a more realistic framework for understanding decision-making under uncertainty when learning is involved.
• CMAB heuristics, particularly the Softmax rule, effectively model human learning and adaptation in signal detection.
• This CMAB approach offers principled solutions for classical SDT problems where initial information is incomplete.