Asynchronous Deep Double Dueling Q-learning for trading-signal execution in limit order book markets

Area of Science:

• Quantitative Finance
• Machine Learning
• Algorithmic Trading

Background:

• High-frequency trading (HFT) relies on sophisticated algorithms to execute orders rapidly.
• Limit order books (LOBs) present complex dynamics that challenge traditional trading strategies.
• Reinforcement learning (RL) offers a powerful framework for developing adaptive trading agents.

Purpose of the Study:

• To develop and evaluate a deep reinforcement learning agent for optimal limit order placement in HFT.
• To assess the agent's performance in a realistic simulated NASDAQ equity trading environment.
• To investigate the agent's ability to manage inventory and maximize trading returns.

Main Methods:

• Utilized the ABIDES limit order book simulator to create a realistic trading environment.
• Developed an RL agent using Deep Dueling Double Q-learning with APEX architecture.
• Trained the agent on historical order book data and synthetic alpha signals with varying noise levels.

Main Results:

• The RL agent learned an effective trading strategy for inventory management and order placement.
• The agent demonstrated superior performance compared to a heuristic benchmark strategy.
• The approach proved robust even with noisy trading signals.

Conclusions:

• Deep reinforcement learning is a viable approach for creating adaptive and profitable HFT strategies.
• RL agents can effectively learn optimal order placement policies in complex LOB environments.
• This study highlights the potential of RL for advancing automated trading systems.