Affiliation:
1. University of Waterloo
2. ETH Zürich
3. Google, Inc
Abstract
We study the problem of finding and monitoring fixed-size subgraphs in a continually changing large-scale graph. We present the first approach that (i) performs worst-case optimal computation and communication, (ii) maintains a total memory footprint linear in the number of input edges, and (iii) scales down per-worker computation, communication, and memory requirements linearly as the number of workers increases, even on adversarially skewed inputs.
Our approach is based on worst-case optimal join algorithms, recast as a data-parallel dataflow computation. We describe the general algorithm and modifications that make it robust to skewed data, prove theoretical bounds on its resource requirements in the
massively parallel computing
model, and implement and evaluate it on graphs containing as many as 64 billion edges. The underlying algorithm and ideas generalize from finding and monitoring subgraphs to the more general problem of computing and maintaining relational equi-joins over dynamic relations.
Subject
General Earth and Planetary Sciences,Water Science and Technology,Geography, Planning and Development
Cited by
47 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Attempts in Worst-Case Optimal Joins on Relational Data Systems: A Literature Survey;2023 IEEE 6th International Conference on Cloud Computing and Artificial Intelligence: Technologies and Applications (CloudTech);2023-11-21
2. Research on Query Task Fragmentation in the Scenario of Storage and Compute Separation;2023 International Conference on Intelligent Computing and Next Generation Networks(ICNGN);2023-11-17
3. Distributed Subgraph Query Processing Using Filtering Scores on Spark;Electronics;2023-08-29
4. Locality Sensitive Hashing for Optimizing Subgraph Query Processing in Parallel Computing Systems;Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining;2023-08-04
5. Efficient GPU-Accelerated Subgraph Matching;Proceedings of the ACM on Management of Data;2023-06-13