Valence precedes value in neural encoding of prediction error

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Psychiatry

Background:

• Event-related potentials (ERPs) following feedback in reinforcement learning tasks are thought to represent neural encoding of prediction errors.
• Prior research indicates that between 240-340 ms, multiple prediction error encodings, including value (signed quantitative error) and valence (signed error), co-occur.
• Existing methods for distinguishing these encodings are unreliable.

Approach:

• A meta-analysis of reinforcement learning experiments was conducted, primarily involving monetary rewards and losses.
• Value and valence encodings were identified using conventional difference wave methodology.
• Bayesian analysis incorporating nulls was employed to analyze the predicted behavior of each encoder, enabling full discrimination.

Key Points:

• A valence encoding, responsive only to immediate trial outcomes, precedes a value encoding sensitive to the broader experimental context.
• The study differentiates between value and valence encodings of prediction errors in reinforcement learning.
• Bayesian analysis provided a more robust method for discriminating between value and valence signals.

Conclusions:

• The findings suggest a temporal dissociation between valence and value encodings of prediction errors in reinforcement learning.
• This sequential processing has significant implications for computational models of human reinforcement learning.
• Understanding these distinct neural signals can refine models of decision-making and learning.