Affiliation:
1. University of California
Abstract
Big spatial data has become ubiquitous, from mobile applications to satellite data. In most of these applications, data is continuously growing to huge volumes. Existing systems for big spatial data organize records at either the record-level or block-level. Systems that use record-level structures include key-value stores and LSM-Tree stores, which support insert and delete operations and they are optimized for highly-selective queries. On the other hand, systems like GeoSpark that use block-level structures (e.g. 128 MB each) are more efficient for analytical queries, but they cannot incrementally maintain the partitioned data and do not support delete operations. This paper proposes a general framework that enables block-level systems to incrementally maintain spatial partitions, in the presence of bulk insertions and deletions, in distributed file system (DFS) blocks. We first formally study the incremental spatial partitioning problem for big data and demonstrate its NP-hardness. Then, we propose a cost model to estimate the performance of queries on the partitioned data and the effect of modifying it as the data grows. After that, we provide three different implementations of the incremental partitioning framework. Comprehensive experiments on large real datasets show that our proposed partitioning algorithms outperforms state-of-the-art spatial partitioning methods.
Publisher
Association for Computing Machinery (ACM)
Subject
General Earth and Planetary Sciences,Water Science and Technology,Geography, Planning and Development
Reference63 articles.
1. accumulo [n.d.]. Apache Accumulo. https://accumulo.apache.org/. Visisted on 15-Sep-2021. accumulo [n.d.]. Apache Accumulo. https://accumulo.apache.org/. Visisted on 15-Sep-2021.
2. Sort-based query-adaptive loading of R-trees
3. Selectivity estimation in spatial databases
4. Storage management in AsterixDB
5. AsterixDB
Cited by
5 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Analysis of Geospatial Data Loading;Proceedings of the Tenth International Workshop on Testing Database Systems;2024-06-09
2. Structural Designs Meet Optimality: Exploring Optimized LSM-tree Structures in a Colossal Configuration Space;Proceedings of the ACM on Management of Data;2024-05-29
3. Tinba: Incremental partitioning for efficient trajectory analytics;Advanced Engineering Informatics;2023-08
4. Learned Spatial Data Partitioning;Proceedings of the Sixth International Workshop on Exploiting Artificial Intelligence Techniques for Data Management;2023-06-18
5. Efficient distributed algorithms for distance join queries in spark-based spatial analytics systems;International Journal of General Systems;2023-02-19