A new regularized and robust LSTM learning algorithm for industrial soft sensors

Abstract

Modern industrial processes often have nonlinearity, multivariate, time-delay, and measurement outliers, which make accurate data-driven modeling of key performance indicators difficult. To address these issues, a robust and regularized long short-term memory (LSTM) neural network for soft sensors in complex industrial processes was proposed. First, a conventional LSTM architecture was used as the basic model to deal with nonlinearity and time delay. Thereafter, a novel LSTM loss function that combines the excellent resistance to outliers of Huber M-Loss with the superior model reduction capability of ℓ1 regularization was designed. Subsequently, a backpropagation through time training algorithm for the proposed LSTM was developed, including the chain derivative calculation and updating formulas. The adaptive moment estimation was applied to perform the gradient update, while the grid search and moving window cross-validation were used to find the optimal hyperparameters. Finally, nonlinear artificial datasets with time series and outliers, as well as an industrial dataset of a desulfurization process, were applied to investigate the performance of the proposed soft sensor. Simulation results show that the proposed algorithm outperforms other state-of-the-art soft sensors in terms of predictive accuracy and training time. The causal relationship of the data-driven soft sensor trained by the proposed algorithm is consistent with the field operation and chemical reactions of the desulfurization process.