Pitfalls in quantifying exploration in reward-based motor learning and how to avoid them.

Area of Science:

• Motor control and learning
• Computational neuroscience
• Robotics

Background:

• Motor learning often involves binary feedback (success/failure).
• Post-failure variability may indicate exploration for improved performance.
• Existing methods struggle to reliably quantify exploration from trial-to-trial changes.

Purpose of the Study:

• To investigate the reliability of estimating exploration from trial-to-trial changes in reward-based motor learning.
• To identify and address pitfalls in current exploration quantification methods.
• To develop a robust method for model-free quantification of exploration.

Main Methods:

• Simulated reward-based motor learning using four existing models.
• Analyzed trial-to-trial changes in movement variability following success and failure.
• Developed and applied the additional trial-to-trial change (ATTC) method.

Main Results:

• Simple post-failure change estimates were sensitive to learner and task parameters.
• Identified two key pitfalls: correlated noise/exploration and exploration in reference trials.
• The ATTC method reliably estimated exploration for models with binary, previous-trial-dependent exploration.

Conclusions:

• Quantifying exploration in motor learning requires careful consideration of feedback loops and reference points.
• The ATTC method offers a reliable, model-free approach for specific exploration strategies.
• This method necessitates a substantial number of trials for accurate estimation.