Hierarchical control over foraging behavior by anterior cingulate cortex.

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Neuroscience

Background:

• Foraging behavior involves sequential decision-making to optimize reward and cost.
• The anterior cingulate cortex (ACC) is implicated in foraging, but its precise role is debated.
• Reinforcement learning (RL) models increasingly characterize foraging, with complexity scaling with task demands.

Purpose of the Study:

• To review existing RL foraging models and their hierarchical structures.
• To propose a novel framework where ACC guides foraging using model-based hierarchical reinforcement learning.
• To elucidate the hierarchical organization of ACC function along a rostral-caudal gradient.

Main Methods:

• Review of reinforcement learning foraging models.
• Conceptual extension of RL models to incorporate hierarchical structures.
• Proposal of a hierarchical functional organization of the ACC.

Main Results:

• Foraging problems often exhibit hierarchical structures.
• The ACC's role in foraging can be understood through a hierarchical RL framework.
• A rostral-caudal gradient in ACC function is proposed: rostral for high-level goals, midcingulate for subgoals and actions.

Conclusions:

• The anterior cingulate cortex (ACC) likely implements foraging decisions through a hierarchical reinforcement learning mechanism.
• ACC's hierarchical organization, from rostral (goals) to midcingulate (actions), provides a new perspective on its function in foraging.
• This framework unifies previous theories and offers a scalable approach to understanding foraging computation.