Metraipolis › archive › dc_bf8ce431f8d1
Behavioral Compositionality of Sparse Autoencoder Representations — arXiv:2610.21261
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.21261
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T09:12:04.068Z
- SHA-256 of the archived text
ed2246048d604f5fb11833af57e6e8689ee9d16be4a233cb30d99099d913d999- Stable URL of this page
https://metraipolis.com/documento/dc_bf8ce431f8d1
How to cite
Behavioral Compositionality of Sparse Autoencoder Representations — arXiv:2610.21261 — arXiv (en), https://arxiv.org/abs/2610.21261 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bf8ce431f8d1 (Metraipolis archive dc_bf8ce431f8d1, sha256 ed2246048d604f5fb11833af57e6e8689ee9d16be4a233cb30d99099d913d999)
BibTeX
@misc{dcbf8ce431f8d1,
title = {Behavioral Compositionality of Sparse Autoencoder Representations — arXiv:2610.21261},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.21261},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bf8ce431f8d1; sha256 ed2246048d604f5fb11833af57e6e8689ee9d16be4a233cb30d99099d913d999},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21261. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Sparse autoencoders (SAEs) represent model activations with sparse, linearly decoded features, but the downstream effects of those features need not compose additively. We introduce SAE Additivity Residuals (SAR), which measure the behavioral variation left unexplained above a chosen interaction order. Under independent feature subset sampling, population SAR is exactly the normalized energy of interactions above that order and bounds coalition ordering error for the optimal low order approximation. To distinguish compositionality associated with the SAE representation from that induced by downstream computation, we compare learned SAE coordinates with sum preserving random rotations of the same selected subspace and find lower first order residuals for the learned coordinates across all 18 model and behavior settings. The coordinates are nevertheless not fully additive: pairwise terms improve held out coalition ordering in every setting and outperform parameter matched models using randomly selected third order terms. In a combined steering experiment, pairwise modeling raises rank fidelity from 0.5874 to 0.7790 and reduces ordering error from 0.1444 to 0.0606. Together, these results show that SAE coordinates support comparatively compositional behavioral descriptions while faithful joint explanations and interventions still require interactions beyond individual feature effects. Code is available at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.21261 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T08:44:17.928Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).