Deep Reinforcement Learning: Reservoir Optimization from Pixels

Abstract

The application of Artificial Intelligence (AI) methods in the petroleum industry gain traction in recent years. In this paper, Deep Reinforcement Learning (RL) by pixel data is used to maximize the Net Present Value (NPV) of waterflooding by changing the water injection rate. This work is the first step towards showing that learning from pixel information offers many benefits, for example, understanding the reservoir's physics directly by measuring changes in pressure and saturation distribution without taking into consideration reservoir petrophysical property and the total amount of wells with corresponding locations. The optimization is tested on the 2D model, which is a vertical section of the SPE 10 model. It has been shown that RL is able to optimize waterflooding in a 2D compressible reservoir with the 2-phase flow (oil-water) by means of pixel data. In the first few thousands of updates, optimization remains in the baseline NPV since it takes more time to converge from raw pixel data than to use classical well production/injection rate information. The reservoir NPV increased by 15 percent as a result of optimization, where the optimum scenario results in less watercut and more stable production. Additionally, we evaluated the impact of choosing the action set for optimization and examined two cases where water injection well can change injection pressure with a step of 200 psi and 600 psi. The results show that in the second case, AI optimization is exploiting the limitation of the reservoir simulator and tries to imitate a cycled injection regime, which results in a 7% higher NPV than the first case.