Learning to be confident: How agents learn confidence based on prediction errors

Area of Science:

• Cognitive psychology
• Neuroscience
• Machine learning

Background:

• Decision confidence ideally reflects the probability of a correct choice.
• The learning mechanisms for calibrating confidence remain unclear.

Purpose of the Study:

• To investigate how individuals learn to calibrate their decision confidence.
• To determine if humans use prediction errors to update confidence computations.

Main Methods:

• A perceptual decision-making experiment with manipulated feedback regimes (positive vs. negative).
• Analysis of confidence ratings and objective performance across feedback conditions.
• Modeling confidence using a single-layer neural network based on prediction errors.

Main Results:

• Confidence ratings dynamically tracked feedback, increasing with positive feedback and decreasing with negative feedback.
• Objective performance was unaffected by the feedback manipulation.
• A neural network model incorporating prediction errors provided a better fit to behavioral data than a valence-based model.

Conclusions:

• Human confidence computation is a dynamic process, updated based on trial-level prediction errors.
• This updating mechanism is statistically efficient and specific to confidence, not objective performance.
• Findings support a learning principle analogous to training function approximators using prediction errors.