Abstract
As a popular spatial operation, the k-Nearest Neighbors (kNN) query is widely used in various spatial application systems. How to efficiently process a kNN query on spatial big data has always been an important research topic in the field of spatial data management. The centralized solutions are not suitable for spatial big data due to their poor scalability, while the existing distributed solutions are not efficient enough to meet the high real-time requirements of some spatial applications. Therefore, we introduce the Proportional Integral Derivative (PID) control technology into kNN query processing and propose a PID-based kNN query processing algorithm (PIDKNN) for spatial big data based on Spark. In this algorithm, the whole data space is divided into grid cells of the same size using the grid partition method, and the grid-based index is constructed. On this basis, the grid-based density peak clustering algorithm is used to cluster spatial data, and the corresponding PID parameters are set for each cluster. When performing kNN queries, the PID algorithm is used to estimate the radius growth step size of kNN queries, thereby realizing kNN query processing with a variable query radius growth step based on a feedback mechanism, which greatly improves the efficiency of kNN query processing. A series of experimental results show that the PIDKNN algorithm has good performance and scalability and is superior to the existing parallel kNN query processing methods.
Funder
National Key R&D Program of China
National Natural Science Foundation of China
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference32 articles.
1. Efficient Parallel kNN Joins for Large Data in MapReduce;Chi;Proceedings of the International Conference on Extending Database Technology,2012
2. Improving the Performance of kNN in the MapReduce Framework Using Locality Sensitive Hashing
3. Research on Spatial Data Index and kNN Query Technology under Big Data;Dong;M.D. Thesis,2013
4. Geospark: A Cluster Computing Framework for Processing Large-Scale Spatial Data;Yu;Proceedings of the SIGSPATIAL International Conference on Advances in Geographic Information Systems,2015
5. Spark sql: Relational Data Processing in Spark;Armbrust;Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD),2015
Cited by
2 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献