Evaluation of Stock Closing Prices using Transformer Learning

Abstract

Predicting stock markets remains a critical and challenging task due to many factors, such as the enormous volume of generated price data, instant price data changes, and sensitivity to human sentiments, wars, and natural disasters. Since the previous three years of the COVID-19 pandemic, forecasting stock markets is more difficult, complex, and problematic for stock market analysts. However, technical analysts of the stock market and academic researchers are continuously trying to develop innovative and modern methods for forecasting stock market prices, using statistical techniques, machine learning, and deep learning-based algorithms. This study investigated a Transformer sequential-based approach to forecast the closing price for the next day. Ten sliding window timesteps were used to forecast next-day stock closing prices. This study aimed to investigate reliable techniques based on stock input features. The proposed Transformer-based method was compared with ARIMA, Long-Short Term Memory (LSTM), and Random Forest (RF) algorithms, showing its outstanding results on Yahoo Finance data, Facebook Intra data, and JPMorgan's Intra data. Each model was evaluated using Mean Absolute Error (MSE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE).