Metraipolis › archive › dc_efc8cdefbd39

First-Order Steering: Translating Weight Adaptation into Activation Steering — arXiv:2610.04283

Source
arXiv
Original
https://arxiv.org/abs/2610.04283
License (as archived)
CC0-1.0
Archived
2026-10-06T05:26:06.403Z
SHA-256 of the archived text
62f7477db05fe4b32ae313e2a6c310def233c72be16ffd257052835c120c9100
Stable URL of this page
https://metraipolis.com/documento/dc_efc8cdefbd39

How to cite

First-Order Steering: Translating Weight Adaptation into Activation Steering — arXiv:2610.04283 — arXiv, https://arxiv.org/abs/2610.04283 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_efc8cdefbd39 (Metraipolis archive dc_efc8cdefbd39, sha256 62f7477db05fe4b32ae313e2a6c310def233c72be16ffd257052835c120c9100)
BibTeX
@misc{dcefc8cdefbd39,
  title = {First-Order Steering: Translating Weight Adaptation into Activation Steering — arXiv:2610.04283},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.04283},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_efc8cdefbd39; sha256 62f7477db05fe4b32ae313e2a6c310def233c72be16ffd257052835c120c9100},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.04283. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Activation steering exploits interpretable directions in the residual stream to enable inference-time manipulation of model behavior. Composing steering vectors to apply multiple target behaviors simultaneously is important in various fields-including AI alignment and safety-but remains a challenge for existing activation steering methods. In contrast, prior work in model merging shows that target behaviors represented by learned weight adaptations can be combined with high accuracy. A method that translates weight adaptations into activation steering vectors could therefore extend prior work in model merging to generate composable steering vectors that better enable simultaneous inference-time behavioral control. For this, we introduce First-Order Steering, a formulation of activation steering as a first-order approximation of weight update matrices parameterized by a vector of steering strengths, and establish theoretical bounds on the approximation error of first-order steering. We then develop a novel model merging procedure, HeRD-Merging, which minimizes the first-order approximation error terms to enable higher first-order steering accuracy. Together, our method produces steering vectors that control both individual and composed behaviors more accurately than existing activation steering methods. Furthermore, HeRD-Merging matches the performance of conventional model-merging baselines, while producing weight adaptations that admit more accurate first-order steering vectors.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.04283 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:05.825Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).