Practical Application of Deep Reinforcement Learning to Optimal Trade Execution
Author:
Byun Woo Jae1ORCID, Choi Bumkyu1ORCID, Kim Seongmin1, Jo Joohyun1
Affiliation:
1. Qraft Technologies, Inc., 3040 Three IFC, 10 Gukjegeumyung-ro, Yeongdeungpo-gu, Seoul 07326, Republic of Korea
Abstract
Although deep reinforcement learning (DRL) has recently emerged as a promising technique for optimal trade execution, two problems still remain unsolved: (1) the lack of a generalized model for a large collection of stocks and execution time horizons; and (2) the inability to accurately train algorithms due to the discrepancy between the simulation environment and real market. In this article, we address the two issues by utilizing a widely used reinforcement learning (RL) algorithm called proximal policy optimization (PPO) with a long short-term memory (LSTM) network and by building our proprietary order execution simulation environment based on historical level 3 market data of the Korea Stock Exchange (KRX). This paper, to the best of our knowledge, is the first to achieve generalization across 50 stocks and across an execution time horizon ranging from 165 to 380 min along with dynamic target volume. The experimental results demonstrate that the proposed algorithm outperforms the popular benchmark, the volume-weighted average price (VWAP), highlighting the potential use of DRL for optimal trade execution in real-world financial markets. Furthermore, our algorithm is the first commercialized DRL-based optimal trade execution algorithm in the South Korea stock market.
Reference24 articles.
1. Nevmyvaka, Y., Feng, Y., and Kearns, M. (2006, January 25–29). Reinforcement learning for optimized trade execution. Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA, USA. 2. Hendricks, D., and Wilcox, D. (2014, January 27–28). A reinforcement learning extension to the Almgren-Chriss framework for optimal trade execution. Proceedings of the 2014 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr), London, UK. 3. Optimal execution of portfolio transactions;Almgren;J. Risk,2001 4. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv. 5. Dabérius, K., Granat, E., and Karlsson, P. (2019, April 21). Deep Execution-Value and Policy Based Reinforcement Learning for Trading and Beating Market Benchmarks. Available online: https://ssrn.com/abstract=3374766.
|
|