R-DDQN: Optimizing Algorithmic Trading Strategies Using a Reward Network in a Double DQN
-
Published:2024-05-22
Issue:11
Volume:12
Page:1621
-
ISSN:2227-7390
-
Container-title:Mathematics
-
language:en
-
Short-container-title:Mathematics
Author:
Zhou Chujin1ORCID, Huang Yuling1, Cui Kai1ORCID, Lu Xiaoping1ORCID
Affiliation:
1. School of Computer Science and Engineering, Macau University of Science and Technology, Macao, China
Abstract
Algorithmic trading is playing an increasingly important role in the financial market, achieving more efficient trading strategies by replacing human decision-making. Among numerous trading algorithms, deep reinforcement learning is gradually replacing traditional high-frequency trading strategies and has become a mainstream research direction in the field of algorithmic trading. This paper introduces a novel approach that leverages reinforcement learning with human feedback (RLHF) within the double DQN algorithm. Traditional reward functions in algorithmic trading heavily rely on expert knowledge, posing challenges in their design and implementation. To tackle this, the reward-driven double DQN (R-DDQN) algorithm is proposed, integrating human feedback via a reward function network trained on expert demonstrations. Additionally, a classification-based training method is employed for optimizing the reward function network. The experiments, conducted on datasets including HSI, IXIC, SP500, GOOGL, MSFT, and INTC, show that the proposed method outperforms all baselines across six datasets and achieves a maximum cumulative return of 1502% within 24 months.
Funder
Science and Technology Development Fund, Macau SAR
Reference42 articles.
1. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv. 2. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11–17). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada. 3. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA. 4. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv. 5. Language models are few-shot learners;Brown;Adv. Neural Inf. Process. Syst.,2020
|
|