Metraipolis › archive › dc_deef10469f22
The Independence Prior of SAEs Fragments Visual Concepts — arXiv:2610.04112
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.04112
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:24:18.378Z
- SHA-256 of the archived text
7fc369ab92be95ed55d82ce97d358a300d0a20de12aa49f09fc867fb50bc8220- Stable URL of this page
https://metraipolis.com/documento/dc_deef10469f22
How to cite
The Independence Prior of SAEs Fragments Visual Concepts — arXiv:2610.04112 — arXiv, https://arxiv.org/abs/2610.04112 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_deef10469f22 (Metraipolis archive dc_deef10469f22, sha256 7fc369ab92be95ed55d82ce97d358a300d0a20de12aa49f09fc867fb50bc8220)
BibTeX
@misc{dcdeef10469f22,
title = {The Independence Prior of SAEs Fragments Visual Concepts — arXiv:2610.04112},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.04112},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_deef10469f22; sha256 7fc369ab92be95ed55d82ce97d358a300d0a20de12aa49f09fc867fb50bc8220},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.04112. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Sparse Autoencoders (SAEs) decompose model activations into sparse combinations of interpretable dictionary atoms. Although SAEs are grounded in the Linear Representation Hypothesis (LRH), their objective smuggles in an additional prior: concepts across patches are treated as independent, an assumption clearly violated by natural images and by the activations they induce. We therefore specialize LRH to vision through the Markov-Field Linear Representation Hypothesis (MFLRH), which adds the missing spatial dependencies to the LRH assumptions. We thus propose Spatial-SAE as an amortized MAP estimator under the MFLRH. Spatial-SAE consistently outperforms standard SAEs in concept recovery and interpretability. Across four variants, it achieves a 96% average win rate on synthetic concept recovery and improves interpretability on DINOv2 activations, at a reconstruction cost concentrated in high spatial frequencies. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.04112 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:24:05.428Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).