Metraipolis › archive › dc_94105e830d16
CivicBench: A Benchmark and Memory Framework for Web-Based Civic Data Analytics — arXiv:2611.01145
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.01145
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T10:31:43.403Z
- SHA-256 of the archived text
5b7aacb42f175691a13c0fb146e8895de1f9110e8d9690d054dafc8c2c403394- Stable URL of this page
https://metraipolis.com/documento/dc_94105e830d16
How to cite
CivicBench: A Benchmark and Memory Framework for Web-Based Civic Data Analytics — arXiv:2611.01145 — arXiv (en), https://arxiv.org/abs/2611.01145 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_94105e830d16 (Metraipolis archive dc_94105e830d16, sha256 5b7aacb42f175691a13c0fb146e8895de1f9110e8d9690d054dafc8c2c403394)
BibTeX
@misc{dc94105e830d16,
title = {CivicBench: A Benchmark and Memory Framework for Web-Based Civic Data Analytics — arXiv:2611.01145},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.01145},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_94105e830d16; sha256 5b7aacb42f175691a13c0fb146e8895de1f9110e8d9690d054dafc8c2c403394},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01145. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Municipalities publish service requests, geographic boundaries, weather observations, demographic estimates, and operational guidance through independently maintained Web portals. Civic analysis must combine these sources while preserving their schemas, source links, spatial and temporal scopes, and interpretive limits. Tool-using language agents can perform this integration, but existing task benchmarks rarely test whether a computed claim remains supported by the records used. We present CivicBench, a 400-task benchmark over four cities and two years that materializes civic Web data into frozen bundles for reproducible evaluation. Executable contracts score computation, source linkage, coverage, scope, qualified answers, and refusal when the requested conclusion is not supported. We also introduce Task-State Memory (TSM), which compiles tool observations into typed assertions, reconciles records only within compatible scopes, and retrieves a bounded state organized around unresolved constraints. Across nine completed model pairs, eight yield positive mean-score differences with 95\% semantic-contract-cluster bootstrap intervals excluding zero; the ninth interval crosses zero. The benchmark and memory framework make the records behind a claim inspectable at the point of decision. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.01145 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T10:29:48.498Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).