Metraipolis › archive › dc_752039aebdc8

CivicBench: A Benchmark and Memory Framework for Web-Based Civic Data Analytics — arXiv:2611.01145

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.01145
License (as archived)
CC0-1.0
Archived
2026-11-03T10:52:31.985Z
SHA-256 of the archived text
aead533663898a1f8555b2be26934b14f9d4b42b3f079c5b1e5edc40559eddc8
Stable URL of this page
https://metraipolis.com/documento/dc_752039aebdc8

How to cite

CivicBench: A Benchmark and Memory Framework for Web-Based Civic Data Analytics — arXiv:2611.01145 — arXiv (en), https://arxiv.org/abs/2611.01145 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_752039aebdc8 (Metraipolis archive dc_752039aebdc8, sha256 aead533663898a1f8555b2be26934b14f9d4b42b3f079c5b1e5edc40559eddc8)
BibTeX
@misc{dc752039aebdc8,
  title = {CivicBench: A Benchmark and Memory Framework for Web-Based Civic Data Analytics — arXiv:2611.01145},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01145},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_752039aebdc8; sha256 aead533663898a1f8555b2be26934b14f9d4b42b3f079c5b1e5edc40559eddc8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01145. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Municipalities publish service requests, geographic boundaries, weather observations, demographic estimates, and operational guidance through independently maintained Web portals. Civic analysis must combine these sources while preserving their schemas, source links, spatial and temporal scopes, and interpretive limits. Tool-using language agents can perform this integration, but existing task benchmarks rarely test whether a computed claim remains supported by the records used. We present CivicBench, a 400-task benchmark over four cities and two years that materializes civic Web data into frozen bundles for reproducible evaluation. Executable contracts score computation, source linkage, coverage, scope, qualified answers, and refusal when the requested conclusion is not supported. We also introduce Task-State Memory (TSM), which compiles tool observations into typed assertions, reconciles records only within compatible scopes, and retrieves a bounded state organized around unresolved constraints. Across nine completed model pairs, eight yield positive mean-score differences with 95\% semantic-contract-cluster bootstrap intervals excluding zero; the ninth interval crosses zero. The benchmark and memory framework make the records behind a claim inspectable at the point of decision.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.01145 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:49:38.293Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).