Metraipolis › archive › dc_bc09e78e563d
Look Before You Steer: Geometry Predicts SAE Feature Steerability — arXiv:2609.22782
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.22782
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T07:06:48.213Z
- SHA-256 of the archived text
b03fe51af50a71e84366a477f999a13d42c958a43e1f83ee16064bed69f78a94- Stable URL of this page
https://metraipolis.com/documento/dc_bc09e78e563d
How to cite
Look Before You Steer: Geometry Predicts SAE Feature Steerability — arXiv:2609.22782 — arXiv (en), https://arxiv.org/abs/2609.22782 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bc09e78e563d (Metraipolis archive dc_bc09e78e563d, sha256 b03fe51af50a71e84366a477f999a13d42c958a43e1f83ee16064bed69f78a94)
BibTeX
@misc{dcbc09e78e563d,
title = {Look Before You Steer: Geometry Predicts SAE Feature Steerability — arXiv:2609.22782},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.22782},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bc09e78e563d; sha256 b03fe51af50a71e84366a477f999a13d42c958a43e1f83ee16064bed69f78a94},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22782. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Steering with SAE features requires per-feature coefficient tuning, which currently demands intervention sweeps. We ask whether properties of the SAE itself, computable before any forward pass, predict which features will be cheap or expensive to steer. We show that variation in SAE feature steerability is partially predicted by decoder-space geometry: neighbor density and maximum cosine similarity to nearby decoder directions, both computable from the SAE weight matrix before any intervention, rank features by how much steering they require for a fixed behavioral effect ($\rho$ up to $-0.546$, $p < 10^{-6}$, AUROC 0.610-0.822 across conditions; the signal is rank-based, consistent with grid discreteness). This geometry-steerability relationship replicates across two Gemma-2 model scales (2B and 9B), two SAE widths (16K and 65K), and is detectable cross-architecturally on Llama-3.1-8B-Instruct ($\rho = -0.266$, $n = 300$). On Qwen3-8B with BatchTopK SAEs, geometry predicts whether a feature is steerable at all but not the continuous ordering among responsive features, revealing a boundary condition tied to SAE training regime. The signal weakens at deep proportional layer depth in both models, where the cost of steering exceeds our intervention budget, a consistent depth boundary. These results provide preliminary evidence that pre-steering geometry can partially inform coefficient selection, offering a path toward screening features for controllability before deployment.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.22782 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:58:09.217Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).