Metraipolis › archive › dc_ca5160ea3df9
Cross-Architecture Steering Transfer in Language Models: A Systematic Empirical Study — arXiv:2608.05164
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.05164
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-25T21:27:02.485Z
- SHA-256 of the archived text
44ac4f369410898d353f95f1234d50beedb5250731ea393bc8c68c14c4829406- Stable URL of this page
https://metraipolis.com/documento/dc_ca5160ea3df9
How to cite
Cross-Architecture Steering Transfer in Language Models: A Systematic Empirical Study — arXiv:2608.05164 — arXiv (en), https://arxiv.org/abs/2608.05164 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ca5160ea3df9 (Metraipolis archive dc_ca5160ea3df9, sha256 44ac4f369410898d353f95f1234d50beedb5250731ea393bc8c68c14c4829406)
BibTeX
@misc{dcca5160ea3df9,
title = {Cross-Architecture Steering Transfer in Language Models: A Systematic Empirical Study — arXiv:2608.05164},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.05164},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ca5160ea3df9; sha256 44ac4f369410898d353f95f1234d50beedb5250731ea393bc8c68c14c4829406},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.05164. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Independently trained large language models may develop shared internal representations of semantic concepts despite architectural differences -- but whether this geometric similarity has functional consequences for cross-model behavioural control remains untested. We present the first systematic evaluation of cross-model steering transfer and show that shared LLM geometry is functionally exploitable, conditionally: concept directions from one model can steer a different independently trained model when sufficient representational capacity exists. We study five open-weight models spanning three parameter scales (0.8B--8B) and two architectural lineages, training one Sparse Autoencoder per model across 15 semantic domains and testing alignment across all 20 directed model pairs. We observe a suggestive discontinuity near 1.7B parameters: at >= 1.7B scale, 47--49% of cross-model feature pairs validate (Pearson r >= 0.60, Procrustes cosines 0.895--0.956), while alignment degrades sharply below 0.8B. Cross-model steering vectors (B3-TI) achieve a 71.0% win rate across 15 supervised concepts versus 68.0% for same-model native vectors; a single universal vector achieves 67.3% in 4 of 5 models without any per-model supervision. Transfer degrades for models below 1.7B and for one model with generation instability, confirming that functional exploitability requires sufficient representational capacity. Our findings underscore the importance of scale thresholds in mechanistic interpretability: tools validated at 7B scale may not transfer to smaller models without revalidation. We provide the first functional complement to the Platonic Representation Hypothesis -- geometric convergence across independently trained LLMs supports cross-model behavioural control without fine-tuning, under the identified scale conditions. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-07 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.05164 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-25T21:24:38.173Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).