Metraipolis › archive › dc_7372d2fa056a

When Harness Beats Scale, and When Reading Beats Both — arXiv:2609.34366

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.34366
License (as archived)
CC0-1.0
Archived
2026-10-07T21:02:27.679Z
SHA-256 of the archived text
5680498d0c114a3d1130ab85458b6a7024122858e703e9b546d2b474ac2c62d8
Stable URL of this page
https://metraipolis.com/documento/dc_7372d2fa056a

How to cite

When Harness Beats Scale, and When Reading Beats Both — arXiv:2609.34366 — arXiv (en), https://arxiv.org/abs/2609.34366 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7372d2fa056a (Metraipolis archive dc_7372d2fa056a, sha256 5680498d0c114a3d1130ab85458b6a7024122858e703e9b546d2b474ac2c62d8)
BibTeX
@misc{dc7372d2fa056a,
  title = {When Harness Beats Scale, and When Reading Beats Both — arXiv:2609.34366},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.34366},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7372d2fa056a; sha256 5680498d0c114a3d1130ab85458b6a7024122858e703e9b546d2b474ac2c62d8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34366. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We describe our system for DocSem, the document-grounded quantitative reasoning shared task at DocInsights 2026, and analyze why it succeeded on labeled data and failed on the test set. The pipeline pairs hybrid block retrieval with Program-of-Thoughts (PoT) generation executed in a sandboxed interpreter, self-consistency sampling, and entity enrichment from chunk-level knowledge graphs. On our held-out split, application architecture moved the metrics far more than model scale did: PoT added 0.282 joint accuracy to a compact 7B model but at most 0.005 to a 72B model, and a 27B model with the full harness matched the 72B (0.884 vs.\ 0.873) at roughly 2.7$\times$ fewer parameters and a quarter of the CO$_2$. We read this through a distinction between world knowledge, which scales steeply with parameters, and language knowledge, which scales gently, and show that structured-output training makes a compact model harness-ready rather than merely small. On the raster, watermarked test PDFs the same system collapsed to 13.58\% joint (rank 149 of 163); a controlled re-rendering of the validation set reproduces the OCR half of the collapse while bounding what the simulation misses. Auditing the physical nature of evaluation inputs precedes architecture, and the leaderboard's bimodality is consistent with reading quality, not reasoning, having separated the field.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Information Retrieval (cs.IR).
Pagina del lavoro: https://arxiv.org/abs/2609.34366 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:02.962Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).