Action-specialized expert ensemble trading system with extended discrete action space using deep reinforcement...

Area of Science:

• * Quantitative Finance
• * Machine Learning
• * Algorithmic Trading

Background:

• * Existing reinforcement learning (RL) trading systems require performance enhancements.
• * Current ensemble methods lack action-specific optimization.

Purpose of the Study:

• * To propose a novel action-specialized expert ensemble method for RL trading.
• * To improve the efficiency and robustness of RL-based trading strategies.
• * To evaluate the impact of different reward functions on trading performance.

Main Methods:

• * Developed action-specialized expert models for 'buy,' 'hold,' and 'sell' actions.
• * Defined reward values correlated with specific actions and market conditions.
• * Compared proposed system's profits against single and common ensemble trading systems.
• * Analyzed performance across S&P500, Hang Seng Index, and Eurostoxx50 data.
• * Tested sensitivity using profit, Sharpe ratio, and Sortino ratio reward functions.

Main Results:

• * The proposed model demonstrated 39.1% and 21.6% greater efficiency than single and common ensemble models, respectively.
• * Expanding the action space from 3 to 11 and 21 actions increased cumulative returns by 427.2% and 856.7%.
• * Models trained with Sharpe and Sortino ratios outperformed those using profit alone, with Sortino ratio showing a slight edge.

Conclusions:

• * The action-specialized expert ensemble method offers a significant improvement for RL trading systems.
• * The approach enhances trading efficiency, robustness, and cumulative returns.
• * Reward function selection impacts performance, with risk-adjusted ratios like Sortino and Sharpe proving effective.