From deterministic to stochastic: an interpretable stochastic model-free reinforcement learning framework for...

Area of Science:

• Algorithmic Trading
• Quantitative Finance
• Machine Learning

Background:

• Portfolio optimization is crucial for maximizing returns in algorithmic trading.
• Reinforcement learning (RL) offers advanced sequential decision-making capabilities for trading.
• Noisy financial data challenges deterministic RL strategies in portfolio management.

Purpose of the Study:

• To develop an interpretable stochastic reinforcement learning framework for portfolio optimization.
• To address the challenges posed by noisy financial data in achieving profitable portfolios.
• To balance risk and return using a novel risk-aware reward function.

Main Methods:

• Reconstruction of benchmark deterministic and stochastic reinforcement learning algorithms.
• Introduction of a risk-aware reward function to balance portfolio risk and return.
• Proposal of a novel interpretable stochastic reinforcement learning framework with Gaussian Mixture parameterized policies and quantile-based distributional critics.

Main Results:

• The proposed algorithm achieved a 63.1% annual rate of return on U.S. market stocks.
• Market value maximum drawdown was reduced by 10% during back-testing, including the March 2020 stock market crash.
• Demonstrated superior performance compared to benchmark algorithms in portfolio optimization.

Conclusions:

• The novel interpretable stochastic reinforcement learning framework effectively enhances portfolio optimization.
• The risk-aware reward function and distributional critic contribute to improved risk-return balance.
• The proposed method offers a robust solution for profitable algorithmic trading with noisy financial data.