Metraipolis › archive › dc_aa6947bf30c0
Uncovering Uncontrolled Repetition through Residual Stream Dynamics — arXiv:2609.38802
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.38802
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T12:44:17.623Z
- SHA-256 of the archived text
00550c880d8f972ed40c6c9aadfcef6422a946c326503f561696a962ae742241- Stable URL of this page
https://metraipolis.com/documento/dc_aa6947bf30c0
How to cite
Uncovering Uncontrolled Repetition through Residual Stream Dynamics — arXiv:2609.38802 — arXiv (en), https://arxiv.org/abs/2609.38802 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aa6947bf30c0 (Metraipolis archive dc_aa6947bf30c0, sha256 00550c880d8f972ed40c6c9aadfcef6422a946c326503f561696a962ae742241)
BibTeX
@misc{dcaa6947bf30c0,
title = {Uncovering Uncontrolled Repetition through Residual Stream Dynamics — arXiv:2609.38802},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.38802},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aa6947bf30c0; sha256 00550c880d8f972ed40c6c9aadfcef6422a946c326503f561696a962ae742241},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38802. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Uncontrolled repetition can prolong autoregressive generation in large language models (LLMs) and enable resource consumption attacks. Prior analyses of repetitive generation have identified strongly activated features in intermediate and late layers. However, how uncontrolled repetition activity emerges and develops before becoming prominent in these layers remains insufficiently understood. In this paper, we investigate this question primarily in large vision-language models (LVLMs), which support a richer set of uncontrolled repetitions through both visual and textual inputs. We propose Tokenwise Residual Comparison (TRC), a method that identifies and localizes anomalies associated with repetition from residual dynamics during generation. TRC compares attention and multilayer perceptron writes to the residual stream across generated tokens to identify patterns associated with repetition. It then selectively suppresses coordinates in the residual stream at the identified layer. Experiments show that TRC effectively mitigates uncontrolled repetition, reducing loop rates by 57\% on average. Our analysis further shows that repetition semantics emerge in shallow layers and propagate through the residual stream, disrupting normal representations. TRC also generalizes to large language models (LLMs) and large reasoning models (LRMs), where it consistently captures analogous repetition dynamics and achieves effective mitigation. Our work broadens the study of repetitive generation from its prominent internal representations to earlier opportunities for intervention, providing insights for mitigating resource consumption attacks. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.38802 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T12:14:13.943Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).