Generalizing Bulk-Synchronous Parallel Processing for Data Science: From Data to Threads and Agent-Based Simulations-Reference-Cited by-同舟云学术

Generalizing Bulk-Synchronous Parallel Processing for Data Science: From Data to Threads and Agent-Based Simulations

Published:2023-06-13 Issue:2 Volume:1 Page:1-28
ISSN:2836-6573
Container-title:Proceedings of the ACM on Management of Data
language:en
Short-container-title:Proc. ACM Manag. Data

Author:

Tian Zilu¹^ORCID,Lindner Peter¹^ORCID,Nissl Markus²^ORCID,Koch Christoph¹^ORCID,Tannen Val³^ORCID

Affiliation:

1. EPFL, Lausanne, Switzerland

2. TU Wien, Vienna, Austria

3. University of Pennsylvania, Philadelphia, PA, USA

Abstract

We generalize the bulk-synchronous parallel (BSP) processing model to make it better support agent-based simulations. Such simulations frequently exhibit hierarchical structure in their communication patterns which can be exploited to improve performance. We allow for the creation of temporary artificial network partitions during which agents synchronize only locally within their group in a way that does not compromise the correctness of a simulation. We have built a distributed engine, CloudCity, which uses this idea to improve the locality of computation, communication, and synchronization in such simulations. We experimentally evaluate the performance of our system on a benchmark of simulation workloads and compare it against other popular BSP-like systems, obtaining insights into the impact of various system design choices and optimization on simulation engine performance.

Publisher

Association for Computing Machinery (ACM)

Link

https://dl.acm.org/doi/pdf/10.1145/3589296

Reference72 articles.

1. David Adam. 2020. Special report: The simulations driving the world's response to COVID-19. https://www.nature.com/articles/d41586-020-01003--6 David Adam. 2020. Special report: The simulations driving the world's response to COVID-19. https://www.nature.com/articles/d41586-020-01003--6

2. Akka Developers. 2011. Akka Documentation. https://akka.io/ Akka Developers. 2011. Akka Documentation. https://akka.io/

3. Apache Giraph Developers. 2011. Apache Giraph. https://giraph.apache.org/ Apache Giraph Developers. 2011. Apache Giraph. https://giraph.apache.org/

4. Apache Spark Developers. 2018. Apache Spark. https://spark.apache.org Apache Spark Developers. 2018. Apache Spark. https://spark.apache.org

5. Kiyoshi Arai , Hiroshi Deguchi , and Hiroyuki Matsui . 2006. Agent-based modeling meets gaming simulation . Vol. 2 . Springer Science & Business Media , Kisarazu, Chiba , Japan. Kiyoshi Arai, Hiroshi Deguchi, and Hiroyuki Matsui. 2006. Agent-based modeling meets gaming simulation. Vol. 2. Springer Science & Business Media, Kisarazu, Chiba, Japan.

Cited by 2 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Multi-Stage Vertex-Centric Programming for Agent-Based Simulations;Proceedings of the 22nd ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences;2023-10-22

2. Scalable parallel and distributed simulation of an epidemic on a graph;PLOS ONE;2023-09-29