Predicting the Performance of Retail Market Firms: Regression and Machine Learning Methods

Abstract

The problem of predicting profitability is exceptionally relevant for investors and company owners. This paper examines the factors affecting firm performance and tests and compares various methods based on linear and non-linear dependencies between variables for predicting firm performance. In this study, the methods include random effects regression, individual machine learning algorithms with optimizers (DNN, LSTM, and Random Forest), and advanced machine learning methods consisting of sets of algorithms (portfolios and ensembles). The training sample includes 551 retail-oriented companies and data for 2017–2019 (panel data, 1653 observations). The test sample contains data for these companies for 2020. This study combines two approaches (stages): an econometric analysis of the influence of factors on the company’s profitability and machine learning methods to predict the company’s profitability. To compare forecasting methods, we used parametric and non-parametric predictive measures and ANOVA. The paper shows that previous profitability has a strong positive impact on a firm’s performance. We also find a non-linear positive effect of sales growth and web traffic on firm profitability. These variables significantly improve the prediction accuracy. Regression is inferior in forecast accuracy to machine learning methods. Advanced methods (portfolios and ensembles) demonstrate better and more steady results compared with individual machine learning methods.