Long-Term, Multi-Variate Production Forecasting Using Non-Stationary Transformer

Abstract

Abstract Petroleum production forecasting plays an important role in business decisions related to field development planning. Machine learning and artificial intelligence have been used extensively in recent years as they are capable of interpreting and analyzing complex data. In particular, transformers have been used in long-term forecasting of time-series data because of their global-range modeling ability. In this work, non-stationary transformer is used to forecast long-term production in order to address issues with ‘vanilla’ transformer, such as joint distribution change over time. Data-driven model is developed using non-stationary transformer which has two main modules namely series stationarization and de-stationary attention. Series stationarization unifies the statistics of each input and converts the output with restored statistics for better predictability. To address over-stationarization problem, de-stationary attention is devised to recover intrinsic non-stationary information into temporal dependencies by approximating distinguishable attention from raw series. Stationarization improves series predictability, whereas de-stationary attention enhances model capability. Non-stationary transformers can hence be used to effectively learn from long-time series data. Non-stationary transformer is used to forecast production for Olympus benchmark model which has 11 production wells and 7 water injection wells with 20 years simulation horizon. Multi-variate dataset is created with oil and water production rates for producers, and water injection rate for injectors. Thus, training dataset has 29 time-series with monthly data for 20 years period, first 70% of which is used for training while 15% each are used for validating and testing the model. Non-stationary transformer is used to develop data-driven model for forecasting, and results are compared with ‘vanilla’ transformer. The model takes previous four months of data as input, and outputs next four months values. Vanilla transformer gives an order of magnitude higher mean squared error (MSE) during the training period as compared to non-stationary transformer. This difference is even bigger in the test period, where vanilla transformer gives two orders of magnitude higher MSE. Performance of vanilla transformer deteriorates in test period as it is unable to learn non-stationarity prevalent in the dataset, while non-stationary transformer gives similar performance in both training and test period. Next, we develop a surrogate model using non-stationary transformer for ensemble of 10 realizations. Dataset includes 290 time-series with 29 for each of 10 realizations. The Surrogate model is able to maintain similar performance as compared to single realization case, showing that it could be used for real world cases with hundreds of wells. Non-stationary transformer is used to create data-driven, long-term prediction model for oilfield production. Series stationarization helps learn non-stationarity in the time series, while de-stationary attention helps it to recover non-stationary attention. Thus, the model can better learn the dynamical system and outperform vanilla transformer model.