Abstract
Motivated by performance optimization of large-scale graph processing systems that distribute the graph across multiple machines, we consider the balanced graph partitioning problem. Compared to most of the previous work, we study the
multi-dimensional
variant in which balance according to multiple weight functions is required. As we demonstrate by experimental evaluation, such multi-dimensional balance is essential for achieving performance improvements for typical distributed graph processing workloads.
We propose a new scalable technique for the multidimensional balanced graph partitioning problem. It is based on applying randomized projected gradient descent to a non-convex continuous relaxation of the objective. We show how to implement the new algorithm efficiently in both theory and practice utilizing various approaches for the projection step. Experiments with large-scale graphs containing up to hundreds of billions of edges indicate that our algorithm has superior performance compared to the state of the art.
Subject
General Earth and Planetary Sciences,Water Science and Technology,Geography, Planning and Development
Cited by
13 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. FSM: A Fine-Grained Splitting and Merging Framework for Dual-Balanced Graph Partition;Proceedings of the VLDB Endowment;2024-05
2. TeraHAC: Hierarchical Agglomerative Clustering of Trillion-Edge Graphs;Proceedings of the ACM on Management of Data;2023-11-13
3. An Open-Source Constraints-Driven General Partitioning Multi-Tool for VLSI Physical Design;2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD);2023-10-28
4. GMaglev: Graph-friendly Consistent Hashing for Distributed Social Graph Partition;2023 2nd International Conference on Innovations and Development of Information Technologies and Robotics (IDITR);2023-05
5. Graph Unlearning;Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security;2022-11-07