High-Frequency Trading Meets Online Learning

Abstract

We propose an optimization framework for market-making in a limit order book, based on the theory of stochastic approximation. The idea is to take advantage of the iterative nature of the process of updating bid and ask quotes in order to make the algorithm optimize its strategy on a trial-and-error basis (i.e., online learning) using a variation of the stochastic gradient-descent algorithm. An advantage of this approach is that the exploration of the system by the algorithm is performed in run-time, so explicit specifications of price dynamics are not necessary, as is the case in the stochastic-control approach [(Gueant et al., 2013, Dealing with the Inventory Risk: A Solution to the Market Making Problem, Mathematics and Financial Economics 7(4), 477–507)]. For price/liquidity modeling, we consider a discrete-time variant of the Avellaneda–Stoikov model [(Avellaneda, M. and S. Stoikov, 2008, Liquidation in Limit Order Books with Controlled Intensity, Mathematical Finance 24(4), 627–650)] similar to its developent in the paper of Laruelle et al. [(Laruelle et al., 2013, Optimal Posting Price of Limit Orders: Learning by trading, Mathematics and Financial Economics 7(3), 359–403)] in the context of optimal liquidation tactics. Our aim is to set the ground for more advanced reinforcement learning techniques and to argue that the rationale of our method is generic enough to be extended to other classes of trading problems besides market-making.