按年被提到
20192019 · 2 次2026
系统、数据库与网络›分布式数据与 CRDT·arXiv:1907.08302 · cs.PF

Quantitative Impact Evaluation of an Abstraction Layer for Data Stream Processing Systems

Guenter Hesse, Christoph Matthies, Kelvin Glass, Johannes Huegle, Matthias Uflacker · 2019
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2019-11-19 · 最近 2020-03-05
2不同的人
3活跃月份
3讨论串
54被引(OpenAlex)
arXiv 摘要(候选层不写导读)

With the demand to process ever-growing data volumes, a variety of new data stream processing frameworks have been developed. Moving an implementation from one such system to another, e.g., for performance reasons, requires adapting existing applications to new interfaces. Apache Beam addresses these high substitution costs by providing an abstraction layer that enables executing programs on any of the supported streaming frameworks. In this paper, we present a novel benchmark architecture for comparing the performance impact of using Apache Beam on three streaming frameworks: Apache Spark Streaming, Apache Flink, and Apache Apex. We find significant performance penalties when using Apache Beam for application development in the surveyed systems. Overall, usage of Apache Beam for the examined streaming applications caused a high variance of query execution times with a slowdown of up to a factor of 58 compared to queries developed without the abstraction layer. All developed benchmark artifacts are publicly available to ensure reproducible results.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论faizshah · 2019-11

“So I have a bad history with Cloud Dataflow. I like BigQuery and Flink a lot, but a lot of companies seem to be switching over to Cloud Dataflow. In my experience Cloud Dataflow's cost is really unpredictable and DataPrep's pipelines (on DataFlow) were far more inefficient compared to an equivalent DataProc Spark pipeline, Spark on Compute Engine, or jus…”

Cloud Dataflow SQL and Cloud Dataflow Batch Flexible Resource Scheduling Release原评论 ↗
评论faizshah · 2019-12

“Check this reference out: https://arxiv.org/pdf/1907.08302.pdf “ Our benchmark results show that Apache Beam has a noticeable impact on the performance of DSPSs in almost all cases. Programs developed using Apache Beam suffered from a slowdown of up to a factor of 58 in the worst case. At the same time, there is one scenario where the query developed usi…”

Google Brass Set 2023 as Deadline to Beat Amazon, Microsoft in Cloud原评论 ↗
被提交为 story · 1 次