Metraipolis › archive › dc_612d9ea8763d

The Parser Already Knows: Lightweight Bias Correction in Constrained Decoding — arXiv:2608.10137

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.10137
License (as archived)
CC0-1.0
Archived
2026-10-19T06:14:10.116Z
SHA-256 of the archived text
dd154539f5f3c08ac89f86047fba34c609c84c4cfe67520be004ad4e5f8c3962
Stable URL of this page
https://metraipolis.com/documento/dc_612d9ea8763d

How to cite

The Parser Already Knows: Lightweight Bias Correction in Constrained Decoding — arXiv:2608.10137 — arXiv (en), https://arxiv.org/abs/2608.10137 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_612d9ea8763d (Metraipolis archive dc_612d9ea8763d, sha256 dd154539f5f3c08ac89f86047fba34c609c84c4cfe67520be004ad4e5f8c3962)
BibTeX
@misc{dc612d9ea8763d,
  title = {The Parser Already Knows: Lightweight Bias Correction in Constrained Decoding — arXiv:2608.10137},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.10137},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_612d9ea8763d; sha256 dd154539f5f3c08ac89f86047fba34c609c84c4cfe67520be004ad4e5f8c3962},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.10137. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Grammar Constrained Decoding (GCD) forces Language Models (LMs) to produce syntactically valid outputs by masking out non-conforming tokens at each step. However, because masking only checks whether each token is valid so far, the resulting distribution over complete outputs diverges from the LM's own distribution conditioned on the grammar, biasing generation toward valid but suboptimal outputs. Online sampling can restore this distribution, but only through costly iterative resampling. Our key insight is that the parser and lexer states that GCD tools already maintain carry a strong signal about future grammatical validity. We introduce SHIM, a lightweight, offline-trained correction of the LM's next-token probabilities, conditioned on this syntactic and lexical state together with candidate next tokens. Since GCD tools already compute these states, SHIM leaves the LM itself untouched. Across bit-vector and text-to-SQL grammars, this correction substantially narrows the gap to the LM's grammar-conditioned distribution compared to masking and online sampling, while running at nearly masking's speed. Even a variant that sees only the next token can improve on both baselines, making SHIM usable with GCD tools that do not expose their parser state.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-12 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.10137 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T06:08:50.307Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).