Metraipolis › archive › dc_4d6a593d4ec2

When More Becomes Less: Position-Dependent Repetition Effects in Language Models — arXiv:2608.04021

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.04021
License (as archived)
CC0-1.0
Archived
2026-10-31T11:24:31.403Z
SHA-256 of the archived text
aba4798007915702425e2b5c8d144ca098a19b3979f4ba9dd911227a90556b63
Stable URL of this page
https://metraipolis.com/documento/dc_4d6a593d4ec2

How to cite

When More Becomes Less: Position-Dependent Repetition Effects in Language Models — arXiv:2608.04021 — arXiv (en), https://arxiv.org/abs/2608.04021 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4d6a593d4ec2 (Metraipolis archive dc_4d6a593d4ec2, sha256 aba4798007915702425e2b5c8d144ca098a19b3979f4ba9dd911227a90556b63)
BibTeX
@misc{dc4d6a593d4ec2,
  title = {When More Becomes Less: Position-Dependent Repetition Effects in Language Models — arXiv:2608.04021},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.04021},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4d6a593d4ec2; sha256 aba4798007915702425e2b5c8d144ca098a19b3979f4ba9dd911227a90556b63},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04021. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Cloze-style probes that vary how often a target token appears implicitly assume that more copies of a target affect prediction the same way regardless of where the readout slot sits. We show this assumption fails. Our two-probe design holds a repeated-target prefix fixed and varies only the readout position: the adjacent probe places the slot immediately after the repeated block; the displaced probe places it inside a fresh sentence frame. Adjacent repetition behaves as priming intuition predicts: $P(\text{target})$ climbs with $N$ and plateaus. Displaced repetition produces an inverted-U: $P(\text{target})$ rises to an early peak and then declines as more copies are added. The displaced inverted-U shows a per-word drop with bootstrap CI excluding zero in all 13 open-access encoder and decoder models we test, and replicates across Spanish, Chinese, German, and French in 42 of 42 multilingual cells. A six-condition causal ablation isolates the effect to exact lexical repetition rather than length, generic redundancy, or semantic-neighbour exposure. A frame-pragmatics control rules out an artefact of the readout frame. Internally, per-target-token attention falls with $N$ while the total budget assigned to the repeated block grows in causal LMs but not in the masked LM we probe. Probes that vary repetition count cannot treat the readout position as orthogonal to what they measure.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-06 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.04021 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T11:13:23.151Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).