AP-IO: Asynchronous Pipeline I/O for Hiding Periodic Output Cost in CFD Simulation

Abstract

Computational fluid dynamics (CFD) simulation often needs to periodically output intermediate results to files in the form of snapshots for visualization or restart, which seriously impacts the performance. In this paper, we present asynchronous pipeline I/O (AP-IO) optimization scheme for the periodically snapshot output on the basis of asynchronous I/O and CFD application characteristics. InAP-IO, dedicated background I/O processes or threads are in charge of handling the file write in pipeline mode, therefore the write overhead can be hidden with more calculation than classic asynchronous I/O. We design the framework ofAP-IOand implement it in OpenFOAM, providing CFD users with a user-friendly interface. Experimental results on theTianhe-2supercomputer demonstrate thatAP-IOcan achieve a good optimization effect for the periodical snapshot output in CFD application, and the effect is especially better for massively parallel CFD simulations, which can reduce the total execution time up to about 40%.