Metraipolis › archive › dc_4a5bd2db4dc0

IDEEA: training-free Input-Dependent stEEring via Activation cluster matching — arXiv:2609.02089

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.02089
License (as archived)
CC0-1.0
Archived
2026-10-17T04:16:40.811Z
SHA-256 of the archived text
94477b604042ffa2c79b973b95425b7960f8868eb95d64576b5be1419a6724b6
Stable URL of this page
https://metraipolis.com/documento/dc_4a5bd2db4dc0

How to cite

IDEEA: training-free Input-Dependent stEEring via Activation cluster matching — arXiv:2609.02089 — arXiv (en), https://arxiv.org/abs/2609.02089 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4a5bd2db4dc0 (Metraipolis archive dc_4a5bd2db4dc0, sha256 94477b604042ffa2c79b973b95425b7960f8868eb95d64576b5be1419a6724b6)
BibTeX
@misc{dc4a5bd2db4dc0,
  title = {IDEEA: training-free Input-Dependent stEEring via Activation cluster matching — arXiv:2609.02089},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.02089},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4a5bd2db4dc0; sha256 94477b604042ffa2c79b973b95425b7960f8868eb95d64576b5be1419a6724b6},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.02089. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Steering aligns large language models (LLMs) by injecting a bias into selected activations at inference time, offering a far cheaper alternative to weight-update methods such as supervised fine-tuning or reinforcement learning. However, most existing training-free steering methods are input-independent: a single direction is fitted once and shared across all inputs. This is fundamentally limiting as different inputs occupy different regions of the activation space and admit different optimal steering directions toward the same target concept, much as the gradient with respect to a fixed loss varies from input to input. We close this gap with IDEEA (Input-Dependent stEEring via Activation cluster matching), a training-free framework for input-dependent steering. IDEEA clusters the positive and negative activation supports per attention head, and solves an optimal-matching problem to construct a set of cluster-conditional directions, all about the target concept. At inference time, it picks from this pool of directions and uses the one that best matches the input's own activation for steering. IDEEA aligns the model toward the target concept while preserving the input's original representation, evidence that activations encoding a concept occupy several distinct sub-regions of the representation space rather than a single one. IDEEA improves the truth $\times$ info rate in TruthfulQA by an average of 9.9% (up to 23.5%) over the best input-independent baseline.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-03 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.02089 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T03:26:49.778Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-03?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).