Metraipolis › archive › dc_b14d99e557dc
Minimally Invasive Steering of Language Models — arXiv:2609.30218
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.30218
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T04:28:04.150Z
- SHA-256 of the archived text
2b2374e095010e4724a8682a45ae73127cd98cc34c40de37b5608bf92bd8ba10- Stable URL of this page
https://metraipolis.com/documento/dc_b14d99e557dc
How to cite
Minimally Invasive Steering of Language Models — arXiv:2609.30218 — arXiv (en), https://arxiv.org/abs/2609.30218 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b14d99e557dc (Metraipolis archive dc_b14d99e557dc, sha256 2b2374e095010e4724a8682a45ae73127cd98cc34c40de37b5608bf92bd8ba10)
BibTeX
@misc{dcb14d99e557dc,
title = {Minimally Invasive Steering of Language Models — arXiv:2609.30218},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.30218},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b14d99e557dc; sha256 2b2374e095010e4724a8682a45ae73127cd98cc34c40de37b5608bf92bd8ba10},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30218. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Pre-logit steering adapts a frozen language model to a test-time reward by adding vectors to its final hidden states. Unregularized reward optimization can substantially alter the output distribution and degrade generation quality. We propose Minimally Invasive Steering Vector Optimization (MISVO), which penalizes interventions using the local KL geometry of the induced token distribution. The resulting Fisher quadratic measures distributional sensitivity and admits an analytic gradient computed through matrix--vector products with the frozen language-model head. We derive an exact decomposition of the sequence-level KL gradient into an analytic Fisher term and a suffix score-function term. For a fixed generation horizon, we show that the suffix term is second order in the steering magnitude and that three Fisher surrogates agree with the full KL gradient to first order. MISVO uses the frozen-reference surrogate to optimize position-specific interventions without updating model parameters. Across preference and code-generation tasks on models with approximately 1B--14B parameters, MISVO achieves the highest mean reward in six of seven model--task settings, with diversity and coherence scores close to those of Best-of-N. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.30218 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T03:50:58.250Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).