Affiliation:
1. NTT Software Innovation Center
2. National Institute of Informatics
3. University of Electro-Communications
Abstract
We propose OptIQ, a query optimization approach for iterative queries in distributed environment. OptIQ removes redundant computations among different iterations by extending the traditional techniques of view materialization and incremental view evaluation. First, OptIQ decomposes iterative queries into invariant and variant views, and materializes the former view. Redundant computations are removed by reusing the materialized view among iterations. Second, OptIQ incrementally evaluates the variant view, so that redundant computations are removed by skipping the evaluation on converged tuples in the variant view. We verify the effectiveness of OptIQ through the queries of PageRank and k-means clustering on real datasets. The results show that OptIQ achieves high efficiency, up to five times faster than is possible without removing the redundant computations among iterations.
Subject
General Earth and Planetary Sciences,Water Science and Technology,Geography, Planning and Development
Cited by
11 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Research on Curriculum Design Method of Teaching Resource Library based on Deep Learning Technology;Highlights in Science, Engineering and Technology;2023-04-11
2. The Lannion report on Big Data and Security Monitoring Research;2022 IEEE International Conference on Big Data (Big Data);2022-12-17
3. Parallel Maintenance of Materialized Views in Large-Scale Analytic Platforms;International Journal of Organizational and Collective Intelligence;2022-07-21
4. DBSpinner: Making a Case for Iterative Processing in Databases;2021 IEEE 37th International Conference on Data Engineering (ICDE);2021-04
5. Iterative Query Processing based on Unified Optimization Techniques;Proceedings of the 2019 International Conference on Management of Data;2019-06-25