Exploration and exploitation during sequential search.

Area of Science:

• Cognitive science
• Neuroscience
• Decision theory

Background:

• Skill acquisition often involves learning from feedback after actions are completed.
• This process can be framed as a search problem: finding actions that maximize rewards.
• Balancing exploitation (using known information) and exploration (seeking new information) is crucial for effective learning.

Purpose of the Study:

• To investigate how individuals learn movement trajectories with monetary reward feedback.
• To determine if human learning behavior aligns with optimal search strategies.

Main Methods:

• Participants learned to adjust movement trajectories based on monetary rewards.
• A mathematical model based on decision theory was used to derive the optimal search policy.
• Participant behavior was compared against predictions from an ideal searcher model.

Main Results:

• Human participants' learning of movement trajectories was effectively predicted by the ideal searcher model.
• The model demonstrated that participants optimally combined exploration and exploitation strategies.
• Feedback in the form of monetary reward facilitated this learning process.

Conclusions:

• Skill learning in movement trajectories can be modeled as an optimal search problem.
• Human behavior in this task aligns with a strategy that balances exploiting current knowledge and exploring for better future outcomes.
• Decision theory provides a robust framework for understanding the computational mechanisms underlying skill acquisition.