High-Performance Reservoir Simulation With Parallel ATHOS

Abstract

Abstract One way to reduce uncertainties in reservoir simulation is to use geostatistical fine grids with little or no upscaling, but this approach can lead to excessively large simulations with literally millions of gridblocks. In the case of compositional studies, another way is to increase the number of components. Running this kind of large simulations requires the use of powerful computers with vector or parallel architectures. In general, existing simulators must be restructured to make the use of parallel computers possible and efficient. Reengineering andre-writing of the source code are time and cost consuming for multi-purpose simulators. Previous papers have described the use of MPI (Message Passing Interface) to trigger parallel processing. This approach is delicate to manage as it involves incremental reengineering of the code, which is rather difficult to perform. We present our experience of parallelization of an existing all-purpose reservoir simulator for shared memory platforms, using OpenMP instructions. The paper describes the methodology of parallelization in three incremental steps: first identifying the most CPU time consuming routines. Second, parallelizing these routines. Finally, developing efficient dedicated parallel preconditioned solvers. It is shown that this approach is easier to manage than the MPI approach while being as efficient on shared memory parallel architectures like SGI Origin2000, IBM SP3, or Compaq ES40. Several large simulations, typically one million gridblocks, are presented to show the capabilities of ATHOS simulator on these parallel architectures. Simulations carried out on NEC SX5 are also presented, for comparison with a vector computer. These simulations include the main physical features of the ATHOS simulator used for black-oil, compositional and fractured reservoir studies. Introduction To respond to the ever-growing needs of reservoir simulation studies with vast amounts of data, in a reasonable elapsed time, a parallel full-featured version of our reservoir simulation software was developed. While the code had originally been designed for vector computers, the trend in oil companies for the past years has been to migrate to parallel computers, due to their low-cost architecture and huge memory space. In this context, parallelization of our code became necessary. We imposed the constraint one of generating a single source code for the sequential, parallel and vector machines. Having the same code for these three types of platforms allows to reduce the overall cost and time of conception/development as well as subsequent integration of new options. The new standard of shared memory parallelization: OpenMP [1], turned out to be the most appropriate to our case, because it is portable, and unlike the MPI [2] technology, it allows parallelization of an already vectorized code without deep restructuration and it also allows incremental developments. The majority of the code was parallelized in a simple way. However, this was not the case for the solver and in particular for the solver preconditioners. Instead, it proved necessary to develop special solvers based on the BiCGSTAB (Bi-Conjuguated Gradient Stabilized) algorithm with parallel preconditioners. ATHOS reservoir simulation software ATHOS is a multipurpose simulator. The version described in this paper and used for the tests is the 4.0 commercial full-featured version. ATHOS provides a lot of different physical options from the most routinely used (black-oil, multi-component) to the more specialized ones (thermal, dual permeability, dual porosity, polymers, steam injection). All the simulated cases used a fully implicit numerical formulation, but ATHOS also provides the alternatve of an IMPES scheme when deemed appropriate. The 3D graphics were handled by SimGrid, a tool from the Reservoir Modeling Line software.