Metraipolis › archive › dc_63eff51ff628

Rethinking SAE Evaluation: An Atomic Interpretable Unit Framework Reveals Hidden Polysemanticity and Redundancy — arXiv:2610.30474

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.30474
License (as archived): CC0-1.0
Archived: 2026-10-23T15:14:56.209Z
SHA-256 of the archived text: e5d684dbaf8ef09e844cbec0dba30eb74fc8c60eb054d45173bc677f48a31d66
Stable URL of this page: https://metraipolis.com/documento/dc_63eff51ff628

How to cite

Rethinking SAE Evaluation: An Atomic Interpretable Unit Framework Reveals Hidden Polysemanticity and Redundancy — arXiv:2610.30474 — arXiv (en), https://arxiv.org/abs/2610.30474 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_63eff51ff628 (Metraipolis archive dc_63eff51ff628, sha256 e5d684dbaf8ef09e844cbec0dba30eb74fc8c60eb054d45173bc677f48a31d66)

BibTeX

@misc{dc63eff51ff628,
  title = {Rethinking SAE Evaluation: An Atomic Interpretable Unit Framework Reveals Hidden Polysemanticity and Redundancy — arXiv:2610.30474},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.30474},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_63eff51ff628; sha256 e5d684dbaf8ef09e844cbec0dba30eb74fc8c60eb054d45173bc677f48a31d66},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.30474. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Sparse autoencoders (SAEs) were originally motivated by discovering monosemantic features, a property that characterizes individual features but does not specify what the learned feature set should look like as a whole. As SAEs have increasingly been adopted for attribution, circuit discovery, and intervention, researchers have begun evaluating metrics that describe global properties of the feature set, such as sparsity, dead feature rates, reconstruction fidelity, and disentanglement, a diverse set that extends far beyond monosemanticity. However, this implicit shift beyond monosemanticity has not been formalized as a clear, well-defined objective that comes before these evaluation metrics. Without such an objective, existing metrics remain ad hoc, with their necessity unjustified. In this work, we formalize this ultimate objective by introducing a new paradigm for SAE evaluation: requiring that SAE features, and more generally dictionary-based decompositions of model representations, form an atomic interpretable unit basis, a coordinate system for semantically meaningful explanation. Under this paradigm, we organize existing and new measurements such as DAME (Dominance-Aware Monosemanticity Evaluation) as proxy metrics and introduce a comprehensive benchmarking framework. Evaluating hundreds of SAEs across multiple architectures and model families, we uncover failure modes in which some SAEs appear monosemantic under existing evaluation metrics while remaining functionally polysemantic: the most dominant features, those contributing most to model computation, are systematically less interpretable and more redundant. These failure modes warrant attention from SAE researchers as targets for improved training and architectures, and motivate dominance-aware evaluation as a standard check for practitioners before SAE features are deployed for attribution, circuit tracing, or intervention.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.30474 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-23T15:09:08.522Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).