Adaptive algorithms for shaping behavior

Area of Science:

• Artificial Intelligence
• Animal Behavior
• Machine Learning

Background:

• Behavioral shaping is a common training method for both animals and artificial agents.
• Effective shaping strategies rely on adapting task difficulty based on learner performance.

Purpose of the Study:

• To propose a computational framework for understanding and optimizing shaping strategies.
• To develop an adaptive shaping heuristic for efficient behavior training.

Main Methods:

• A teacher-student framework was developed where an autonomous agent selects tasks based on learner history.
• Monte Carlo planning under uncertainty was used to analyze shaping algorithms.
• An adaptive shaping heuristic was derived and tested on sequence learning and navigation tasks.

Main Results:

• Near-optimal shaping algorithms balance reinforcement and extinction by adapting learning rates.
• The derived heuristic demonstrated near-optimal performance on a sequence learning task.
• The heuristic robustly trained deep reinforcement learning agents in navigation tasks with sparse, delayed rewards.

Conclusions:

• The proposed teacher-student framework offers a general computational approach to behavior shaping.
• Adaptive shaping is effective for training both biological and artificial agents, particularly in complex environments.
• This work lays the foundation for more sophisticated curricula and adaptive learning systems.