Placing Approach-Avoidance Conflict Within the Framework of Multi-objective Reinforcement Learning.

Area of Science:

• Neuroscience
• Computational Psychiatry
• Reinforcement Learning

Background:

• Psychiatric disorders often involve impaired decision-making, particularly during approach-avoidance conflicts.
• Current experimental tasks pair rewards and harms, creating these conflicts within bandit task paradigms.

Purpose of the Study:

• To reframe approach-avoidance conflict tasks as a multi-objective multi-armed bandit problem.
• To disentangle decision processes from learning by defining decision-makers through long-term average outcomes.
• To develop new task designs capable of differentiating between various decision-making models.

Main Methods:

• Framing approach-avoidance conflict tasks as a multi-objective multi-armed bandit.
• Defining decision-makers as limiting sequences of actions to separate decision from learning.
• Characterizing decision-makers by their 'feasible region' – the set of all possible long-term performances.
• Analyzing three example decision-makers based on reinforcement learning models.

Main Results:

• Existing tasks are insufficient for distinguishing between common reinforcement learning-based decision-makers.
• The geometric properties of decision-makers' feasible regions can be used for differentiation.
• New task designs are proposed with geometric structures tailored to better distinguish between models.

Conclusions:

• The proposed multi-objective multi-armed bandit framework offers a novel perspective on approach-avoidance conflict tasks.
• Current experimental paradigms may not adequately capture the nuances of decision-making deficits in psychiatric disorders.
• Future research should leverage refined task designs to improve the modeling and understanding of decision-making behavior.