异步深度双重决斗 Q学习用于限制订单账本市场的交易信号执行
Peer Nagy1, Jan-Peter Calliess1, Stefan Zohren1,2,3
1Department of Engineering Science, Oxford-Man Institute of Quantitative Finance, University of Oxford, Oxford, United Kingdom.
Frontiers in artificial intelligence
|October 11, 2023
概括
深度强化学习 (RL) 训练交易代理人优化限额订单的放置. 在现实的市场模拟中,RL代理有效地管理库存,并优于基准策略.
科学领域:
- 量化金融 量化金融
- 机器学习 机器学习
- 算法交易是一种算法交易.
背景情况:
- 高频交易 (HFT) 依赖于复杂的算法来快速执行订单.
- 极限订单簿 (LOB) 呈现出复杂的动态,挑战了传统的交易策略.
- 强化学习 (RL) 为开发适应性交易代理提供了一个强大的框架.
研究的目的:
- 开发和评估一个深度强化学习代理,以在HFT中进行最佳的极限订单放置.
- 在一个现实的模拟纳斯达克股票交易环境中评估代理商的表现.
- 调查代理人管理库存和最大限度地提高交易回报的能力.
主要方法:
- 利用ABIDES限制订单簿模拟器来创建一个现实的交易环境.
- 使用深度决斗双重Q学习与APEX架构开发了一个RL代理.
- 在历史订单簿数据和带有不同噪音水平的合成α信号上训练了代理人.
主要成果:
- RL代理商学习了一种有效的库存管理和下单的交易策略.
- 与启发式基准策略相比,该代理商表现优越.
- 这种方法即使在有噪音的交易信号的情况下也被证明是稳健的.
结论:
- 深度强化学习是创建适应性和利性高强化学习策略的可行方法.
- 在复杂的LOB环境中,RL代理可以有效地学习最佳的订单放置政策.
- 这项研究强调了RL在推进自动化交易系统方面的潜力.
相关概念视频
02:20Dynamic Equilibrium
51.7K
A reversible chemical reaction represents a chemical process that proceeds in both forward (left to right) and reverse (right to left) directions. When the rates of the forward and reverse reactions are equal, the concentrations of the reactant and product species remain constant over time and the system is at equilibrium. A special double arrow is used to emphasize the reversible nature of the reaction. The relative concentrations of reactants and products in equilibrium systems vary greatly;...
51.7K
01:18Second Order systems II
115
In an underdamped second-order system, where the damping ratio ζ is between 0 and 1, a unit-step input results in a transfer function that, when transformed using the inverse Laplace method, reveals the output response. The output exhibits a damped sinusoidal oscillation, and the difference between the input and output is termed the error signal. This error signal also demonstrates damped oscillatory behavior. Eventually, as the system reaches a steady state, the error diminishes to zero.
115
01:22Basic Continuous Time Signals
216
Basic continuous-time signals include the unit step function, unit impulse function, and unit ramp function, collectively referred to as singularity functions. Singularity functions are characterized by discontinuities or discontinuous derivatives.
The unit step function, denoted u(t), is zero for negative time values and one for positive time values, exhibiting a discontinuity at t=0. This function often represents abrupt changes, such as the step voltage introduced when turning a car's...
The unit step function, denoted u(t), is zero for negative time values and one for positive time values, exhibiting a discontinuity at t=0. This function often represents abrupt changes, such as the step voltage introduced when turning a car's...
216
01:12Observational Learning
190
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
190
01:20Second Order systems I
166
A servo system exemplifies a second-order system, featuring a proportional controller and load elements that ensure the output position aligns with the input position. The relationship between these components is described by a second-order differential equation. Applying the Laplace transform under zero initial conditions yields the transfer function, showing how inputs are converted to outputs in the system.
By reinterpreting the system, one can derive the closed-loop transfer function, which...
By reinterpreting the system, one can derive the closed-loop transfer function, which...
166
01:26Signal and System
672
A signal x(t) is a set of data or a time function representing a variable of interest. Signals typically convey information about a phenomenon, such as atmospheric temperature, humidity, human voice, television images, a dog's bark, or birdsongs. More generally, a signal can be a function of more than one independent variable. For instance, images depend on horizontal and vertical positions and can be regarded as two-dimensional signals. However, this text will focus on one-dimensional...
672