Multi-factor stock trading strategy based on DQN with multi-BiGRU and multi-head ProbSparse self-attention

Abstract

Abstract Reinforcement learning is widely used in financial markets to assist investors in developing trading strategies. However, most existing models primarily focus on simple volume-price factors, and there is a need for further improvement in the returns of stock trading. To address these challenges, a multi-factor stock trading strategy based on Deep Q-Network (DQN) with Multi-layer Bidirectional Gated Recurrent Unit (Multi-BiGRU) and multi-head ProbSparse self-attention is proposed. Our strategy comprehensively characterizes the determinants of stock prices by considering various factors such as financial quality, valuation, and sentiment factors. We first use Light Gradient Boosting Machine (LightGBM) to classify turning points for stock data. Then, in the reinforcement learning strategy, Multi-BiGRU, which holds the bidirectional learning of historical data, is integrated into DQN, aiming to enhance the model’s ability to understand the dynamics of the stock market. Moreover, the multi-head ProbSparse self-attention mechanism effectively captures interactions between different factors, providing the model with deeper market insights. We validate our strategy’s effectiveness through extensive experimental research on stocks from Chinese and US markets. The results show that our method outperforms both temporal and non-temporal models in terms of stock trading returns. Ablation studies confirm the critical role of LightGBM and multi-head ProbSparse self-attention mechanism. The experiment section also demonstrates the significant advantages of our model through the presentation of box plots and statistical tests. Overall, by fully considering the multi-factor data and the model’s feature extraction capabilities, our work is expected to provide investors with more precise trading decision support. Graphical abstract