A Parallel Reservoir Simulator Based On Local Grid Refinement

Abstract

A Parallel Reservoir Simulator Based On Local Grid Refinement John E. Killough, SPE, Dominic Camilleri, SPE, Bruce L. Darlow, SPE, and John A. Foster, Landmark Graphics Corporation Copyright 1997, Society of Petroleum Engineers, Inc. Abstract The following paper discusses a general-purpose reservoir simulator which has been designed to run efficiently in a parallel computing environment. The unique feature of the model is its ability to run efficiently in either serial or parallel modes. This is achieved through the use of local grid refinement to subdivide a model into different domains for each of the parallel CPU's. Levels of local grid refinement are used to define preconditioners for the two parallel solvers in the simulator. The solvers are based on strip and composite grid decompositions with nested factorization. Test problems for the simulator and solvers range from large IMPES, black-oil cases with severe faulting to a fully-implicit seven-component compositional simulation. Model sizes range to more than one million finite difference cells for both black-oil and compositional cases. Application of these algorithms to several examples shows the utility of both solver techniques. The strip preconditioning exhibits high efficiency for small numbers of processors. For more difficult cases involving greater numbers of processors, the composite grid solver shows greater robustness. Excellent parallel efficiencies were observed for all cases. Introduction The attraction of parallel computing to achieve high computing efficiencies has existed for decades. What was generally lacking was the ability to easily port software to the parallel environment. Because of the diverse nature of reservoir simulation, simple parallelization schemes have not been viable for a usable, general-purpose, approach to reservoir modeling. Profiles of computing work load for a typical model often show tens, if not hundreds, of subroutines which are involved in a substantial portion of the calculations. Because of this, major reprogramming of reservoir simulation models is required to achieve high parallel efficiencies. Several papers in the literature discuss techniques which have been used to bring about efficient parallel reservoir simulations. The unique feature of the simulator of this work is its reliance on local grid refinement for parallelization. With local grid refinement the same simulation program can be used to perform simulations either serially on a single processor or in parallel on multiple processors simply through data manipulation. The following article demonstrates the use of local grid refinement for parallelization. Basically, each grid is assigned to a processor. Examples are given in which variation of the processor assignment and the grid refinement can dramatically effect parallel performance. Different preconditioners for the parallel linear equation solvers are also tested. Finally, a load-balancing example utilizes the flexibility of local grid refinement and processor assignment to achieve improved parallel efficiency. Description of the Simulator The commercial reservoir simulator which formed the basis for this study, VIP, is a full-featured reservoir simulation software system for UNIX workstations. The program is a multi-component, three dimensional, three phase reservoir simulator which contains a number of modules sharing a common compositional formulation. Both fully implicit and IMPES model formulations of the differential equations governing conservation and flow in the reservoir may be utilized including the relaxed volume-balance technique. P. 39^