LazyAct: Lazy actor with dynamic state skip based on constrained MDP.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Reinforcement Learning

Background:

• Deep reinforcement learning (DRL) excels at complex decision-making but suffers from high computational costs.
• Current DRL methods require full neural network computation per decision, increasing costs with interactions and agents.

Purpose of the Study:

• Introduce the LazyAct algorithm to reduce inference computations in DRL.
• Maintain policy performance while significantly decreasing computational demands.

Main Methods:

• Developed LazyAct, inspired by human reasoning to bypass non-critical states.
• Integrated a state skipping branch into actor networks.
• Established optimization objectives for single-agent and multi-agent inference using IMPALA and MAPPO frameworks.
• Employed pre-training and fine-tuning for policy network training.

Main Results:

• LazyAct reduced inferences by ~80% in single-agent and ~40% in multi-agent settings.
• Policy performance was sustained comparably to existing methods.
• Significant reductions in task completion time and Floating Point Operations (FLOPs) were observed.

Conclusions:

• LazyAct offers an efficient approach to DRL by reducing computational overhead.
• The algorithm demonstrates practical applicability by lowering resource requirements without sacrificing performance.