Metraipolis › archive › dc_c759eb753156
A Strong Harness Narrows the Model Gap: A Comparative Evaluation for Legislative Document-to-XML Conversion — arXiv:2610.19999
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.19999
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T20:02:51.705Z
- SHA-256 of the archived text
d3c64527ada73067d0b9462ffb111abf116c49ea2cb478fcf8e877058f0d625f- Stable URL of this page
https://metraipolis.com/documento/dc_c759eb753156
How to cite
A Strong Harness Narrows the Model Gap: A Comparative Evaluation for Legislative Document-to-XML Conversion — arXiv:2610.19999 — arXiv (en), https://arxiv.org/abs/2610.19999 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c759eb753156 (Metraipolis archive dc_c759eb753156, sha256 d3c64527ada73067d0b9462ffb111abf116c49ea2cb478fcf8e877058f0d625f)
BibTeX
@misc{dcc759eb753156,
title = {A Strong Harness Narrows the Model Gap: A Comparative Evaluation for Legislative Document-to-XML Conversion — arXiv:2610.19999},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.19999},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c759eb753156; sha256 d3c64527ada73067d0b9462ffb111abf116c49ea2cb478fcf8e877058f0d625f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.19999. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Legal-content reproduction requires matching the source exactly in section hierarchy, verbatim text, citations, and amendment spans; a dropped citation or mislabeled amendment carries into the products lawyers rely on. We report an industrial pipeline converting U.S. state legislation into strict, schema-constrained XML, and hypothesize that its accuracy comes from harness engineering, the deterministic tooling built around a model, more than the model itself. Our controlled ablation crosses three harnesses (zero-shot, a single tool-using agent, and a decomposed multi-agent pipeline) with three Claude models on 153 gold bills from two jurisdictions. The harness narrows the model gap: the weakest model in the full pipeline beats the strongest model alone (0.948 versus 0.806, paired bootstrap p less than 0.001). Because the two agentic harnesses share identical tools, their gap comes from orchestration: decomposition plus the persisted state and per-agent context it brings. Across four scored dimensions, the model spread at zero-shot (0.68 to 0.81) narrows to 0.95 at the full pipeline; the residual sits almost entirely in citation normalization, where the canonical identifier is retrieved from an external authority rather than generated from the weights, so a deterministic lookup tool supplies the exactness that a larger model does not. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.19999 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T19:35:48.389Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).