Metraipolis › archive › dc_bd15db7d2b8b
Beyond SAEs: Consistent Concept Discovery with Sparse Coding — arXiv:2610.23801
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.23801
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T08:42:44.209Z
- SHA-256 of the archived text
542889b77e469b5215768407d3a5c1e4d58c8a05d7fd82fe6a68d59e1c7598b2- Stable URL of this page
https://metraipolis.com/documento/dc_bd15db7d2b8b
How to cite
Beyond SAEs: Consistent Concept Discovery with Sparse Coding — arXiv:2610.23801 — arXiv (en), https://arxiv.org/abs/2610.23801 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bd15db7d2b8b (Metraipolis archive dc_bd15db7d2b8b, sha256 542889b77e469b5215768407d3a5c1e4d58c8a05d7fd82fe6a68d59e1c7598b2)
BibTeX
@misc{dcbd15db7d2b8b,
title = {Beyond SAEs: Consistent Concept Discovery with Sparse Coding — arXiv:2610.23801},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.23801},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bd15db7d2b8b; sha256 542889b77e469b5215768407d3a5c1e4d58c8a05d7fd82fe6a68d59e1c7598b2},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.23801. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Sparse autoencoders (SAEs) explain foundation model representations by decomposing activations into sparse combinations of learned dictionary atoms. However, the learned dictionaries can vary across training runs, limiting the reliability and reproducibility of SAE-based analyses. We propose Consistent Sparse Dictionary Learning (ConSDL), which replaces the standard linear encoder with per-sample sparse coding. This is motivated by our synthetic analysis showing that coefficients from a single linear encoder pass are suboptimal compared with those obtained by per-sample optimization, suggesting that sparse coding is a reliable mechanism for dictionary learning. Across 16 synthetic settings, ConSDL achieves near-perfect dictionary and coefficient recovery. These gains translate to state-of-the-art cross-seed consistency across four vision and language settings, where ConSDL improves dictionary and coefficient consistency by 17.6% and 35.9%, respectively, over the average of six SAE baselines. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.23801 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T08:39:10.097Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).