Implementing a Hardware Agnostic Commercial Black-Oil Reservoir Simulator

Abstract

AbstractCommercial reservoir simulators have traditionally been optimized for parallel computations on central processing units (CPUs). The recent advances in general-purpose graphics processing units (GPUs) have provided a powerful alternative to CPU, presenting an opportunity to significantly reduce run times for simulations. Realizing peak performance on GPU requires that GPU-specific code be written, and also requires that data are laid out sympathetically to the hardware. The cost of copying data between the CPU memory and GPU memory at the time of this writing is egregious. Peak performance will only be realized if this is minimized.In paper Cao et al., 2021, the authors establish approaches to enable a simulator to give excellent performance on a CPU or GPU, with the same simulation result using either hardware. We discuss how their prototype was generalized into high-quality, maintainable code with applicability across a wide range of models.Different parts of a reservoir simulator benefit from different approaches. A modern, object-oriented simulator requires components to handle initialization, property calculation, linearization, linear solver, well and aquifer calculations, field management, and reporting. Each of these areas will present architectural challenges when broadening the scope of the simulator from CPU only to supporting CPU or GPU. We outline these challenges and present the approaches taken to address them. In particular, we discuss the importance of abstracting compute scheduling, testing methods, data storage classes, and associated memory management to a generic framework layer.We have created a high-quality reservoir simulator with the capacity to run on a CPU or GPU with results that match to within a very small tolerance. We present software engineering approaches that enable the team to achieve and maintain this in the future. In addition, we present test outcomes and discuss how to achieve excellent performance.To our knowledge, no simulator capable of both CPU simulation and full GPU simulation (meaning simulation with no copies of full grid-size data for purposes other than reporting) has been presented. We will present novel software approaches used to implement the first such commercial simulator.