Optimizing Attention and Cognitive Control Costs Using Temporally Layered Architectures.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Current reinforcement learning (RL) prioritizes performance over efficiency.
• Biological systems achieve high performance with optimized energy and decision-making.
• RL agents often lack constraints on computational resources.

Purpose of the Study:

• To develop an RL framework that incorporates computational constraints.
• To investigate biologically inspired mechanisms for efficient decision-making.
• To improve RL agent performance under limited energy and time budgets.

Main Methods:

• Proposed a decision-bounded Markov decision process (DB-MDP) to limit decisions and energy.
• Introduced a temporally layered architecture (TLA) with distinct timescales and energy requirements.
• Experimented with existing RL algorithms and the novel TLA in DB-MDP and continuous control environments.

Main Results:

• Existing RL algorithms performed poorly or suboptimally in decision-bounded environments.
• TLA achieved optimal performance in decision-bounded and continuous control tasks.
• TLA matched state-of-the-art performance while significantly reducing computational cost.

Conclusions:

• TLA enables efficient and high-performing RL agents in resource-constrained settings.
• The DB-MDP framework provides a benchmark for energy-aware RL.
• This research paves the way for energy-efficient and time-aware artificial intelligence.