Metraipolis › archive › dc_22e81772d950

When Copying Is Hard: Copy-Constrained Decoding for Exact Span Reproduction — arXiv:2610.28393

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.28393
License (as archived)
CC0-1.0
Archived
2026-10-21T07:09:09.151Z
SHA-256 of the archived text
da61fd1f3cdaa4a200b547e24e1aa455c68fe970712a9405b0ced5a2b290394c
Stable URL of this page
https://metraipolis.com/documento/dc_22e81772d950

How to cite

When Copying Is Hard: Copy-Constrained Decoding for Exact Span Reproduction — arXiv:2610.28393 — arXiv (en), https://arxiv.org/abs/2610.28393 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_22e81772d950 (Metraipolis archive dc_22e81772d950, sha256 da61fd1f3cdaa4a200b547e24e1aa455c68fe970712a9405b0ced5a2b290394c)
BibTeX
@misc{dc22e81772d950,
  title = {When Copying Is Hard: Copy-Constrained Decoding for Exact Span Reproduction — arXiv:2610.28393},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.28393},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_22e81772d950; sha256 da61fd1f3cdaa4a200b547e24e1aa455c68fe970712a9405b0ced5a2b290394c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.28393. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Precisely reproducing a target span verbatim within a long context is a fundamental capability test for LLMs: even when the target string is present in context, models often still drift within the span or fail to stop correctly, revealing limitations in source grounding and boundary control. This capability has broad practical relevance. For instance, code generation often requires embedding sensitive strings such as API keys exactly as they appear in context. In this work, we construct a benchmark to test exact span reproduction ability across contracts, scientific literature, code, and random sequences, and find that current models still make surprisingly frequent errors. To address this, we propose CopyGen, a lightweight copy-constrained plugin for frozen LLMs that turns copying from a byproduct of next-token prediction into a controlled process. Our key idea is to decompose copying into three distinct decisions: when to copy, how far to copy, and when to stop. Concretely, CopyGen first determines whether to enter copy mode, then predicts a tentative copy length, and finally verifies the generated span token by token. During verification, we adaptively adjust the logits of copy continuation tokens to incentivize or suppress continuation, and emit only the longest valid prefix. Across three LLM backbones and four benchmark tasks, CopyGen improves average exact match by 35% relative without model finetuning, and integrates seamlessly with SFT to further improve it by 4%, while substantially accelerating generation. These results suggest that reliable exact copying is not something LLMs obtain for free, but a decoding-time control problem that requires explicit modeling. Code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.28393 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-21T07:05:58.153Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).