Motivated Optimal Developmental Learning for Sequential Tasks Without Using Rigid Time-Discounts.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• Traditional reinforcement learning often relies on non-emergent, symbolic representations like Q-learning.
• Existing models typically focus on current experiences, neglecting the prediction of future events and environmental dynamics.
• Hidden neurons in emergent networks play a crucial role in complex sequential tasks.

Purpose of the Study:

• To model reinforcement learning using emergent representations for hidden neurons in sequential tasks.
• To investigate the influence of neurotransmitters (serotonin, dopamine) on learning in dynamic scenarios.
• To develop a motivated learning model that avoids the greediness of time-discounting in Q-learning.

Main Methods:

• Developed a novel reinforcement learning model utilizing emergent representations, where states are not manually defined.
• Modeled the influence of hidden neurons (Y neurons) on sequential tasks, such as robot navigation.
• Formulated a maximum-likelihood estimation approach for motivated learning, converting complex optimization into linear estimation.

Main Results:

• The new model effectively learns from both current and predicted future experiences, including delayed rewards.
• Serotonin and dopamine systems were shown to significantly accelerate learning in sequential tasks.
• The model successfully handles limited computational resources and learning experience by simplifying optimization.

Conclusions:

• Emergent representations offer a powerful alternative to symbolic methods in reinforcement learning for sequential tasks.
• Neurotransmitter systems like serotonin and dopamine are critical for efficient learning by modulating experience importance.
• This work pioneers the study of reinforcer influences on hidden neurons in emergent, dynamic sequential learning environments.