Metraipolis › archive › dc_32e8ad2ec95a
ChurnBench: A Drift-Aware Benchmark Demonstrating That Refresh Scheduling, Not Cache Age, Governs Staleness in Agentic AI — arXiv:2609.11515
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.11515
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T05:57:37.967Z
- SHA-256 of the archived text
8c0b936cc1c784fa1894e028c156e26ecf8d1a838dda531cc007238d1f7cec72- Stable URL of this page
https://metraipolis.com/documento/dc_32e8ad2ec95a
How to cite
ChurnBench: A Drift-Aware Benchmark Demonstrating That Refresh Scheduling, Not Cache Age, Governs Staleness in Agentic AI — arXiv:2609.11515 — arXiv (en), https://arxiv.org/abs/2609.11515 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_32e8ad2ec95a (Metraipolis archive dc_32e8ad2ec95a, sha256 8c0b936cc1c784fa1894e028c156e26ecf8d1a838dda531cc007238d1f7cec72)
BibTeX
@misc{dc32e8ad2ec95a,
title = {ChurnBench: A Drift-Aware Benchmark Demonstrating That Refresh Scheduling, Not Cache Age, Governs Staleness in Agentic AI — arXiv:2609.11515},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.11515},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_32e8ad2ec95a; sha256 8c0b936cc1c784fa1894e028c156e26ecf8d1a838dda531cc007238d1f7cec72},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.11515. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
In production, agentic systems answer questions over data that lives in several places and keeps changing: licenses are reassigned, users offboarded, prices changed, contracts renewed. Existing retrieval benchmarks freeze the data, so they cannot ask whether an agent's answer is still true, only whether it found the right passage. We present ChurnBench, an open-source benchmark that generates a four-source enterprise data fabric as a timeline rather than a snapshot. Every change is written to an append-only ground-truth ledger, and gold answers are computed from that ledger, never from the live stores. An answer that was correct when its data was retrieved but wrong when evaluated is therefore detected and labeled a freshness error, distinct from a reasoning error; we validate this by resolving ground truth at both timestamps for every case reported. Using the instrument, we find that when a system refreshes on a schedule, cache age does not predict staleness. Across cache ages of 1, 14, and 28 days, freshness errors were 7, 4, and 4, because scheduled refresh bounds staleness by time-to-live, and no TTL lapse was observed in any window. A controlled ablation confirms the mechanism: disabling tiered refresh raises freshness errors from 4 to 45 at 28 days and leaves them identical at one day. The variable a drift benchmark should sweep is therefore TTL configuration against each entity's rate of change, not drift-window length. ChurnBench, the evaluation harness, and all per-error data are released open source. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-11 in cs.SE). Categorie arXiv: Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2609.11515 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T05:45:13.305Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).