Metraipolis › archive › dc_313993d4ddab
Where Does Streaming State Cost Go? A Reproducible Comparison of Flink and Kafka Streams on Kafka — arXiv:2609.28779
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.28779
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T04:50:26.077Z
- SHA-256 of the archived text
c9cc4e1665b479ea48d0119389c375791c884563889f9d41ecb4ad595ae167a8- Stable URL of this page
https://metraipolis.com/documento/dc_313993d4ddab
How to cite
Where Does Streaming State Cost Go? A Reproducible Comparison of Flink and Kafka Streams on Kafka — arXiv:2609.28779 — arXiv (en), https://arxiv.org/abs/2609.28779 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_313993d4ddab (Metraipolis archive dc_313993d4ddab, sha256 c9cc4e1665b479ea48d0119389c375791c884563889f9d41ecb4ad595ae167a8)
BibTeX
@misc{dc313993d4ddab,
title = {Where Does Streaming State Cost Go? A Reproducible Comparison of Flink and Kafka Streams on Kafka — arXiv:2609.28779},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.28779},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_313993d4ddab; sha256 c9cc4e1665b479ea48d0119389c375791c884563889f9d41ecb4ad595ae167a8},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.28779. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
This paper presents a controlled comparison of exactly-once Kafka pipelines implemented with Apache Flink and Kafka Streams, two engines with different state-management architectures. The state management in Flink occurs through checkpoints in external storage, whereas Kafka Streams restores local state by replaying broker changelogs. The experiments evaluate the effects of the two designs on latency, resource consumption, configuration sensitivity, and recover from injected failures. The engines produced expected outputs across 50 correctness trials, five for each engine-workload combination (exact 95% CI 0.929-1.000). The latencies, however, showed differences when tested in 30 minute trials at a fixed rate of 100 events/sec. The measured ingestion to output interval was 4-6 ms for stateless workloads and 4.3-6.2 s for windowed workloads. The median p99 was 1.97-1.97s for stateless workloads and 5.68-9.92s for windowed respectively. Further, a sensitivity study showed that increasing the durability interval from 1,000 to 10,000 ms shifted the latency distributions in Kafka Streams and Flink between stages. Kafka Streams' W3 median p99 T2-T1 increased from 6,085 to 23,155 ms; Flink's T3-T2 increased from 732 to 8,702 ms. Failure injection experiments demonstrate that latency can interfere with incomplete processing behavior. The results show that exactly-once correctness is a necessary but insufficient measure of stream processing performance. Kafka Streams completed 0/5 stateful JVM-kill and 1/5 in each stateful local-volume-loss trials, while corresponding Flink cells completed 5/5. The cost, placement and magnitude depend on state management, architecture, configuration, and measurement location. Accordingly, evaluations of fault-tolerant streaming systems should holistically measure latency, completion, recovery behavior, and correctness. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-25 in cs.DC). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2609.28779 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T04:28:24.108Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).