Metraipolis › archive › dc_07d96d09e2f1
Constrained Decoding Eliminates Structural Failures in Small LLMs but Reveals a Scale-Dependent Semantic Gap — arXiv:2609.23742
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.23742
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:49:55.182Z
- SHA-256 of the archived text
7e217e0bc54ff661ba0562326f6ff352737ee37f558e7fa9fb83bbbc51430abd- Stable URL of this page
https://metraipolis.com/documento/dc_07d96d09e2f1
How to cite
Constrained Decoding Eliminates Structural Failures in Small LLMs but Reveals a Scale-Dependent Semantic Gap — arXiv:2609.23742 — arXiv (en), https://arxiv.org/abs/2609.23742 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_07d96d09e2f1 (Metraipolis archive dc_07d96d09e2f1, sha256 7e217e0bc54ff661ba0562326f6ff352737ee37f558e7fa9fb83bbbc51430abd)
BibTeX
@misc{dc07d96d09e2f1,
title = {Constrained Decoding Eliminates Structural Failures in Small LLMs but Reveals a Scale-Dependent Semantic Gap — arXiv:2609.23742},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.23742},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_07d96d09e2f1; sha256 7e217e0bc54ff661ba0562326f6ff352737ee37f558e7fa9fb83bbbc51430abd},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.23742. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Small open-source large language models (LLMs) in the 0.6B-4B parameter range are increasingly deployed for structured output generation (JSON, function calling, data extraction), yet little is known about how constrained decoding (CD) interacts with model scale in this regime. We benchmark five models from three families across 14 structured-output tasks under three decoding conditions (native, Outlines, XGrammar). We introduce a two-axis evaluation that separates structural correctness (schema validity) from semantic correctness (content accuracy). We find that CD eliminates all structural failures across all models (schema validity: 78.6-92.9% to 100%), but content accuracy reveals a persistent semantic gap that is scale-dependent: type coercion failures are fully CD-rescuable, while instruction-semantic failures (e.g., multi-step function calling) remain CD-resistant. Schema conformance is necessary but not sufficient for semantic correctness; CD's reach ends exactly where schema conformance ends. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.23742 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T06:48:45.006Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).