Sigmoid-weighted linear units for neural network function approximation in reinforcement learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Neural networks are increasingly used as function approximators in reinforcement learning.
• Deep reinforcement learning algorithms like DQN have achieved human-level performance in various domains.
• Traditional reinforcement learning methods with eligibility traces and softmax action selection are being re-evaluated.

Purpose of the Study:

• To propose two new activation functions for neural networks: sigmoid-weighted linear unit (SiLU) and its derivative (dSiLU).
• To demonstrate that traditional on-policy learning with eligibility traces can be competitive with deep reinforcement learning methods like DQN.
• To validate the proposed activation functions and learning approach in challenging game environments.

Main Methods:

• Implementation of SiLU and dSiLU activation functions for neural network approximation.
• Utilizing on-policy learning with eligibility traces (TD(λ) and Sarsa(λ)) and softmax action selection.
• Testing agents in stochastic SZ-Tetris, Tetris, and Atari 2600 games.

Main Results:

• Achieved new state-of-the-art results in Tetris variants using shallow dSiLU network agents with TD(λ) learning.
• Outperformed DQN in the Atari 2600 domain using deep Sarsa(λ) agents with SiLU and dSiLU hidden units.
• Demonstrated the competitiveness of on-policy learning with eligibility traces against DQN without a target network.

Conclusions:

• The proposed SiLU and dSiLU activation functions enhance neural network performance in reinforcement learning.
• On-policy learning with eligibility traces offers a competitive alternative to experience replay-based methods like DQN.
• This research provides effective and efficient approaches for reinforcement learning agents in complex environments.