Metraipolis › archive › dc_d704da1a1bee

Learning a Unified Cross-Model Semantic Dictionary via Gated Bottleneck Sparse Autoencoders — arXiv:2611.10503

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.10503
License (as archived)
CC0-1.0
Archived
2026-11-10T18:42:33.392Z
SHA-256 of the archived text
27930c287ad15be2bf17cffb13cfdcdf750ae352ff9fad1a83102778e022ffe5
Stable URL of this page
https://metraipolis.com/documento/dc_d704da1a1bee

How to cite

Learning a Unified Cross-Model Semantic Dictionary via Gated Bottleneck Sparse Autoencoders — arXiv:2611.10503 — arXiv (en), https://arxiv.org/abs/2611.10503 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d704da1a1bee (Metraipolis archive dc_d704da1a1bee, sha256 27930c287ad15be2bf17cffb13cfdcdf750ae352ff9fad1a83102778e022ffe5)
BibTeX
@misc{dcd704da1a1bee,
  title = {Learning a Unified Cross-Model Semantic Dictionary via Gated Bottleneck Sparse Autoencoders — arXiv:2611.10503},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.10503},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d704da1a1bee; sha256 27930c287ad15be2bf17cffb13cfdcdf750ae352ff9fad1a83102778e022ffe5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.10503. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Foundation models pretrained on similar data distributions tend to learn overlapping semantic structure, but their representations are distinct due to different model architecture, pretraining objectives and protocols. This prevents unified interpretability and cross-model feature reuse. Existing approaches extract interpretable concepts from individual models and align these concepts in a shared coordinate system using cross-reconstruction objectives, but the alignment only applies to latent coordinates while the concepts themselves remain model-specific and cannot be shared or reused. This paper introduces the Gated Bottleneck Sparse Autoencoder (GB-SAE), which casts unified cross-model interpretability as a representation factorization problem and solves it by simultaneously learning a unified shared dictionary and multiple private model-specific dictionaries via a gated bottleneck sparse autoencoder framework. The shared dictionary encodes the cross-model common semantic structure in unified concepts that are reusable across all participating models (and beyond), while each private dictionary comprises residual concepts that capture model-specific biases. We evaluate GB-SAE with distinct foundation models including ViT, SigLIP, DINOv2, and ConvNeXt. GB-SAE demonstrates superior reconstruction capability, unified and faithful interpretability, strong reusability and generalization to unseen models and downstream tasks. The code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2611.10503 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.591Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).