Metraipolis › archive › dc_79e16d31ce32
Sparse Internal Control of Language Models — arXiv:2611.04349
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.04349
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T18:38:39.970Z
- SHA-256 of the archived text
5b21879de5380ed0f47252096e50866773b63825f12047f1d0ac02d3339fc10d- Stable URL of this page
https://metraipolis.com/documento/dc_79e16d31ce32
How to cite
Sparse Internal Control of Language Models — arXiv:2611.04349 — arXiv (en), https://arxiv.org/abs/2611.04349 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_79e16d31ce32 (Metraipolis archive dc_79e16d31ce32, sha256 5b21879de5380ed0f47252096e50866773b63825f12047f1d0ac02d3339fc10d)
BibTeX
@misc{dc79e16d31ce32,
title = {Sparse Internal Control of Language Models — arXiv:2611.04349},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.04349},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_79e16d31ce32; sha256 5b21879de5380ed0f47252096e50866773b63825f12047f1d0ac02d3339fc10d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04349. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Language models are increasingly used in text generation, decision support, and automated interaction, creating a need for localized, reversible, and selective control of their behavior. Activation steering demonstrates that internal interventions can alter the behavior of frozen models, but how to select and evaluate compact, multi-site control interfaces remains an open question. I introduce sparse internal control, a framework for steering target behavior by applying inference-time interventions to a small set of internal model nodes. A local target-controllability formulation characterizes candidate site--direction pairs by their behavioral effects and specifies constraints on target recovery, off-target preservation, and intervention energy. Two complementary selectors instantiate this formulation: Driver-OMP selects nodes for sparse reconstruction of a desired behavioral displacement, while Coverage-Driver favors stable coverage across prompts. I propose a six-axis control-evaluation protocol measuring reachability, signed reversal, dose response, feedback controllability, off-target preservation, and held-out reuse. Exploratory experiments on indirect object identification, MMLU multiple-choice question answering, sentiment steering, and refusal steering provide evidence for the feasibility of steering task-defined behavior through sparse internal interventions. The six-axis evaluation characterizes these interventions beyond target movement, revealing distinct control properties that depend on the task and available direction information. Together, the formulation, reference algorithms, and evaluation protocol provide a foundation for the systematic study and development of internal control methods for language models. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.04349 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T18:32:52.147Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).