Immediate return preference emerged from a synaptic learning rule for return maximization.

Area of Science:

• Behavioral economics
• Neuroscience
• Computational modeling

Background:

• Organisms frequently exhibit a preference for immediate rewards over larger, delayed rewards.
• This immediate return preference is often explained by subjective discounting of future outcomes.
• Existing models may not fully capture this behavior in interactive contexts.

Purpose of the Study:

• To investigate if immediate return preference can emerge from learning rules designed to maximize objective outcomes.
• To explore the role of internal state representation in learning and decision-making.
• To challenge the sufficiency of subjective discounting as the sole explanation for immediate return preference.

Main Methods:

• Utilizing a synaptic learning rule based on temporal difference (TD) learning.
• Simulating learning processes under conditions with and without proper internal state representation.
• Analyzing the outcome maximization performance of the learning rule.

Main Results:

• The temporal difference (TD) learning rule, aimed at outcome maximization, failed to achieve optimal outcomes.
• The learning rule demonstrated an immediate return preference when internal states were inadequately represented.
• This suggests a mechanism for immediate return preference independent of subjective discounting.

Conclusions:

• Immediate return preference can be a consequence of learning mechanisms failing to properly represent internal states.
• Temporal difference (TD) learning, when misapplied, can lead to suboptimal choices resembling immediate reward seeking.
• The study highlights the critical importance of accurate internal state representation for effective objective maximization in learning systems.