A Multi-Scaling Reinforcement Learning Trading System Based on Multi-Scaling Convolutional Neural Networks

Abstract

Advancements in machine learning have led to an increased interest in applying deep reinforcement learning techniques to investment decision-making problems. Despite this, existing approaches often rely solely on single-scaling daily data, neglecting the importance of multi-scaling information, such as weekly or monthly data, in decision-making processes. To address this limitation, a multi-scaling convolutional neural network for reinforcement learning-based stock trading, termed multi-scaling convolutional neural network SARSA (state, action, reward, state, action), is proposed. Our method utilizes a multi-scaling convolutional neural network to obtain multi-scaling features of daily and weekly financial data automatically. This involves using a convolutional neural network with several filter sizes to perform a multi-scaling extraction of temporal features. Multiple-scaling feature mining allows agents to operate over longer time scaling, identifying low stock positions on the weekly line and avoiding daily fluctuations during continuous declines. This mimics the human approach of considering information at varying temporal and spatial scaling during stock trading. We further enhance the network’s robustness by adding an average pooling layer to the backbone convolutional neural network, reducing overfitting. State, action, reward, state, action, as an on-policy reinforcement learning method, generates dynamic trading strategies that combine multi-scaling information across different time scaling, while avoiding dangerous strategies. We evaluate the effectiveness of our proposed method on four real-world datasets (Dow Jones, NASDAQ, General Electric, and AAPLE) spanning from 1 January 2007 to 31 December 2020, and demonstrate its superior profits compared to several baseline methods. In addition, we perform various comparative and ablation tests in order to demonstrate the superiority of the proposed network architecture. Through these experiments, our proposed multi-scaling module yields better results compared to the single-scaling module.