SparkCAD-Reference-Cited by-同舟云学术

SparkCAD

Published:2022-08 Issue:12 Volume:15 Page:3694-3697
ISSN:2150-8097
Container-title:Proceedings of the VLDB Endowment
language:en
Short-container-title:Proc. VLDB Endow.

Author:

Al-Sayeh Hani¹,Jibril Muhammad Attahir¹,Saeed Muhammad Waleed Bin¹,Sattler Kai-Uwe¹

Affiliation:

1. TU Ilmenau, Germany

Abstract

Developers of Apache Spark applications can accelerate their workloads by caching suitable intermediate results in memory and reusing them rather than recomputing them all over again every time they are needed. However, as scientific workflows are becoming more complex, application developers are becoming more prone to making wrong caching decisions, which we refer to as caching anomalies , that lead to poor performance. We present and give a demonstration of Spark Caching Anomalies Detector (SparkCAD) , a developer decision support tool that visualizes the logical plan of Spark applications and detects caching anomalies.

Publisher

Association for Computing Machinery (ACM)

Subject

General Earth and Planetary Sciences,Water Science and Technology,Geography, Planning and Development

Link

https://dl.acm.org/doi/pdf/10.14778/3554821.3554877

Reference20 articles.

1. [n.d.]. Apache Spark Examples. https://github.com/apache/spark/tree/master/examples/src/main/scala/org/apache/spark/examples. Accessed: 2022-07-11. [n.d.]. Apache Spark Examples. https://github.com/apache/spark/tree/master/examples/src/main/scala/org/apache/spark/examples. Accessed: 2022-07-11.

2. [n.d.]. Monitoring and Instrumentation Spark's Applications. https://spark.apache.org/docs/3.1.2/monitoring.html. Accessed: 2022-07-11. [n.d.]. Monitoring and Instrumentation Spark's Applications. https://spark.apache.org/docs/3.1.2/monitoring.html. Accessed: 2022-07-11.

3. [n.d.]. The HiBench Suite. https://github.com/Intel-bigdata/HiBench/tree/v7.1.1. Accessed: 2022-07-11. [n.d.]. The HiBench Suite. https://github.com/Intel-bigdata/HiBench/tree/v7.1.1. Accessed: 2022-07-11.

4. Hani Al-Sayeh , Muhammad Attahir Jibril , Bunjamin Memishi , and Kai-Uwe Sattler . 2022 . Blink: Lightweight Sample Runs for Cost Optimization of Big Data Applications. In European Conference on Advances in Databases and Information Systems (ADBIS). Springer. Hani Al-Sayeh, Muhammad Attahir Jibril, Bunjamin Memishi, and Kai-Uwe Sattler. 2022. Blink: Lightweight Sample Runs for Cost Optimization of Big Data Applications. In European Conference on Advances in Databases and Information Systems (ADBIS). Springer.

5. Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications

Cited by 1 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Blink: Lightweight Sample Runs for Cost Optimization of Big Data Applications;New Trends in Database and Information Systems;2022