Online Tuning of Hyperparameters in Deep LSTM for Time Series Applications

Abstract

Deep learning is one of the most remarkable artificial intelligence trends. It stands behind numerous recent achievements in several domains, such as speech processing, and computer vision, to mention a few. Accordingly, these achievements have sparked great attention to employing deep learning in time series modelling and forecasting. It is known that the deep learning algorithms built on neural networks contain multiple hidden layers, which make the computation of deep neural network challenging and, sometimes, complex. The reason for this complexity is that obtaining an outstanding and consistent result from such deep architecture requires optimizing many parameters known as hyperparameters. Doubtless, hyperparameter tuning plays a critical role in improving the performance of deep learning. This paper proposes an online tuning approach for the hyperparameters of deep long short-term memory (DLSTM) model in a dynamic fashion. The proposed approach adapts to learn any time series based application, particularly the applications that contain streams of data. The experimental results show that the dynamic tuning of the DLSTM hyperparameters performs better than the original static tuning fashion.