HARL-TRADE: A hierarchical adaptive reinforcement learning framework for second-level high-frequency trading

Area of Science:

• Quantitative Finance
• Artificial Intelligence
• Algorithmic Trading

Background:

• High-frequency trading (HFT) requires adaptive strategies for volatile market conditions.
• Existing discrete sub-agent frameworks exhibit limited adaptability due to rigid market condition allocations.

Purpose of the Study:

• To propose a novel hierarchical framework with an attention-based meta-agent for dynamic sub-agent coordination in HFT.
• To improve adaptability and performance in navigating diverse market regimes.

Main Methods:

• Developed a hierarchical framework incorporating an attention-based meta-agent.
• Utilized market embeddings and reinforcement learning for optimal sub-agent weight adjustment.
• Implemented dynamic sub-agent assignment and multi-head attention mechanisms.

Main Results:

• The proposed framework achieved a 42.15% total return and a 4.19 Sharpe ratio on historical HFT data.
• Demonstrated superior performance compared to state-of-the-art baselines.
• Ablation studies confirmed the effectiveness of the dynamic assignment and attention mechanisms.

Conclusions:

• The attention-based hierarchical framework offers superior adaptability and performance in high-frequency trading.
• Dynamic coordination of sub-agents via a meta-agent effectively addresses market volatility and transitions.