go back
go back
Volume 15, No. 12
SparkCAD: Caching Anomalies Detector for Spark Applications
Authors:
Hani Al-Sayeh (TU Ilmenau)* Muhammad Attahir Jibril (TU Ilmenau) Muhammad Waleed Bin Saeed (TU Ilmenau) Kai-Uwe Sattler (TU Ilmenau)
Abstract
Developers of Apache Spark applications can accelerate their workloads by caching suitable intermediate results in memory and reusing them rather than recomputing them all over again every time they are needed. However, as scientific workflows are becoming more complex, application developers are becoming more prone to making wrong caching decisions, which we refer to as caching anomalies, that lead to poor performance. We present and give a demonstration of Spark Caching Anomalies Detector (SparkCAD), a developer decision support tool that visualizes the logical plan of Spark applications and detects caching anomalies.
PVLDB is part of the VLDB Endowment Inc.
Privacy Policy