Parallel Spatial-Data Conversion Engine: Enabling Fast Sharing of Massive Geospatial Data

Abstract

Large-scale geospatial data have accumulated worldwide in the past decades. However, various data formats often result in a geospatial data sharing problem in the geographical information system community. Despite the various methodologies proposed in the past, geospatial data conversion has always served as a fundamental and efficient way of sharing geospatial data. However, these methodologies are beginning to fail as data increase. This study proposes a parallel spatial data conversion engine (PSCE) with a symmetric mechanism to achieve the efficient sharing of massive geodata by utilizing high-performance computing technology. This engine is designed in an extendable and flexible framework and can customize methods of reading and writing particular spatial data formats. A dynamic task scheduling strategy based on the feature computing index is introduced in the framework to improve load balancing and performance. An experiment is performed to validate the engine framework and performance. In this experiment, geospatial data are stored in the vector spatial data defined in the Chinese Geospatial Data Transfer Format Standard in a parallel file system (Lustre Cluster). Results show that the PSCE has a reliable architecture that can quickly cope with massive spatial datasets.