Factor-based deep reinforcement learning for asset allocation: Comparative analysis of static and dynamic beta reward...

Area of Science:

• Quantitative Finance
• Machine Learning
• Computational Economics

Background:

• Traditional asset allocation struggles with market volatility and structural breaks.
• Deep reinforcement learning (DRL) often overlooks factor exposures, crucial for risk and adaptation.
• Factor exposures ([Formula: see text]) influence risk-adjusted payoffs and adaptive investment responses.

Purpose of the Study:

• To develop a Factor-based Deep Reinforcement Learning for Asset Allocation (FDRL) framework.
• To integrate factor sensitivities into DRL state representation and reward design for improved asset allocation.
• To evaluate the performance of different factor-based reward structures in diverse market conditions.

Main Methods:

• Developed an FDRL framework using rolling regressions to estimate factor sensitivities (momentum, volatility, deviation, volume).
• Implemented PPO, SAC, and TD3 algorithms with five reward variants (Sharpe, Sortino, Static-[Formula: see text], Dynamic-[Formula: see text], Momentum-[Formula: see text]).
• Tested the framework across equities, cryptocurrencies, macroeconomic instruments, and mixed portfolios, with extensive robustness checks.

Main Results:

• Factor-based rewards yielded varied but interpretable results across asset classes.
• In equities, Dynamic-[Formula: see text] increased annualized returns to 23-24% and Sharpe ratios to 1.27.
• Cryptocurrencies showed high returns (38-43%) but regime sensitivity; macro instruments benefited from Static-[Formula: see text] stability; mixed portfolios excelled with Momentum-[Formula: see text].

Conclusions:

• The FDRL framework offers a novel approach to asset allocation by integrating factor exposures into DRL.
• Factor-sensitive rewards demonstrate heterogeneous but valuable outcomes, improving performance and risk management.
• The framework reconciles adaptive responsiveness with interpretability and risk discipline in dynamic investment strategies.