SCE-LSTM: Sparse Critical Event-Driven LSTM Model with Selective Memorization for Agricultural Time-Series Prediction

Abstract

In the domain of agricultural product sales and consumption forecasting, the presence of infrequent yet impactful events such as livestock epidemics and mass media influences poses substantial challenges. These rare occurrences, termed Sparse Critical Events (SCEs), often lead to predictions converging towards average values due to their omission from input candidate vectors. To address this issue, we introduce a modified Long Short-Term Memory (LSTM) model designed to selectively attend to and memorize critical events, emulating the human memory’s ability to retain crucial information. In contrast to the conventional LSTM model, which struggles with learning sparse critical event sequences due to its handling of forget gates and input vectors within the cell state, our proposed approach identifies and learns from sparse critical event sequences during data training. This proposed method, referred to as sparse critical event-driven LSTM (SCE-LSTM), is applied to predict purchase quantities of agricultural and livestock products using sharp-changing agricultural time-series data. For these predictions, we collected structured and unstructured data spanning the years 2010 to 2017 and developed the SCE-LSTM prediction model. Our model forecasts monetary expenditures for pork purchases over a one-month horizon. Notably, our results demonstrate that SCE-LSTM provides the closest predictions to actual daily pork purchase expenditures and exhibits the lowest error rates when compared to other prediction models. SCE-LSTM emerges as a promising solution to enhance agricultural product sales and consumption forecasts, particularly in the presence of rare critical events. Its superior performance and accuracy, as evidenced by our findings, underscore its potential significance in this domain.