Neural network compression for reinforcement learning tasks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Real-world Reinforcement Learning (RL) applications, such as in robotics, demand low-latency, energy-efficient, and high-throughput neural network inference.
• Sparsity and pruning are established methods for optimizing neural network inference, improving energy efficiency, latency, and throughput.

Purpose of the Study:

• To systematically investigate the application of pruning and quantization techniques for optimizing neural networks in popular RL algorithms.
• To determine the applicability limits of these optimization methods in RL tasks for hardware deployment.

Main Methods:

• The study applied pruning and quantization to Deep Q-Network (DQN) and Soft Actor Critic (SAC) algorithms.
• Experiments were conducted across diverse RL environments, including MuJoCo and Atari.

Main Results:

• Achieved up to a 400-fold reduction in neural network size.
• Demonstrated the effectiveness of pruning and quantization for optimizing RL inference.

Conclusions:

• Pruning and quantization are effective for optimizing neural networks in RL, enabling hardware deployment.
• These optimizations reduce power consumption and latency while increasing throughput for RL systems.