Metraipolis › archive › dc_705ebd327618
Can Large Language Models Reason about Event-Time Stream-Processing Semantics? — arXiv:2608.12348
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.12348
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T04:17:51.462Z
- SHA-256 of the archived text
8a87ad7d2f444ed3747563cf5eb9413cd19a72a6122120bcdd50307aa12eeef1- Stable URL of this page
https://metraipolis.com/documento/dc_705ebd327618
How to cite
Can Large Language Models Reason about Event-Time Stream-Processing Semantics? — arXiv:2608.12348 — arXiv (en), https://arxiv.org/abs/2608.12348 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_705ebd327618 (Metraipolis archive dc_705ebd327618, sha256 8a87ad7d2f444ed3747563cf5eb9413cd19a72a6122120bcdd50307aa12eeef1)
BibTeX
@misc{dc705ebd327618,
title = {Can Large Language Models Reason about Event-Time Stream-Processing Semantics? — arXiv:2608.12348},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.12348},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_705ebd327618; sha256 8a87ad7d2f444ed3747563cf5eb9413cd19a72a6122120bcdd50307aa12eeef1},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.12348. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Streaming systems increasingly hand work to large language models (LLMs): writing pipelines, triaging alerts, reading logs. All of it assumes the model knows how event-time stream processing behaves, and we test that assumption directly. StreamReason-Bench asks a model to stand in for an event-time stream processor. Given a windowed query and a stream of out-of-order events, it reports which windows fire, with their aggregates, and which events are dropped as late. The answer key comes from a small reference implementation of Dataflow-model semantics, so we can grade exactly, and with a partial-credit row-F1, without running an engine. On 600 generated items covering tumbling, hopping, session, and processing-time windows, the models do poorly on event time. When told to answer directly, no model that actually follows the instruction clears 34% exact match; chain-of-thought (CoT) roughly doubles that for several of them (GPT-4o goes from 0.34 to 0.48), and only one frontier model that reasons by default comes near solving the set (0.85). A processing-time control, with no watermarks and nothing late, is almost solved by every capable model. The gap points to event-time and late-data handling, not windowing or arithmetic, as the hard part. Sorting errors by window type tells the same story: late-data mistakes dominate the event-time windows and vanish on the control, while session windows mostly fail on where the session boundaries fall. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-14 in cs.AI). Categorie arXiv: Databases (cs.DB) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.12348 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T04:13:06.646Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).