Metraipolis › archive › dc_6d9b6840f2dd
Matryoshka attribution: Learning to attribute language model outputs to representations and weights — arXiv:2609.25518
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.25518
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T14:34:28.362Z
- SHA-256 of the archived text
a1769625d7dab24084ac00e04a08d81d3b7e4c360f31b13a668fcbb0e7bf25be- Stable URL of this page
https://metraipolis.com/documento/dc_6d9b6840f2dd
How to cite
Matryoshka attribution: Learning to attribute language model outputs to representations and weights — arXiv:2609.25518 — arXiv (en), https://arxiv.org/abs/2609.25518 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6d9b6840f2dd (Metraipolis archive dc_6d9b6840f2dd, sha256 a1769625d7dab24084ac00e04a08d81d3b7e4c360f31b13a668fcbb0e7bf25be)
BibTeX
@misc{dc6d9b6840f2dd,
title = {Matryoshka attribution: Learning to attribute language model outputs to representations and weights — arXiv:2609.25518},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.25518},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6d9b6840f2dd; sha256 a1769625d7dab24084ac00e04a08d81d3b7e4c360f31b13a668fcbb0e7bf25be},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.25518. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Attributing language model outputs to their internal computations is an open problem in interpretability. Existing methods, which use causal interventions, gradients, or learnable masks, either are infeasibly expensive or struggle to identify actual causally-important internal computations. We propose framing attribution as the problem of identifying nested subsets of internal components which minimise a downstream loss. To learn this task, we introduce Matryoshka Attribution (MAttr), a mask learning method that parametrises the mask with a simple differentiable sigmoid top-$k$ operator. We supervise training over all sparsities simultaneously by randomising $k$ over training, resulting in a learned ordering of components by attribution score. MAttr achieves number 1 on the official leaderboard of the Mechanistic Interpretability Benchmark (Mueller et al., 2025); our method identifies sparse and task-transferrable circuits across varying circuit bases. As a practical application, we show that MAttr can be trained with reinforcement learning to identify weight changes responsible for downstream behaviours in LLM finetuning. We train MAttr on refusal judge scores and find that restoring $1\%$ of Llama 3.1 8B Instruct's weights to their base model state is sufficient to remove refusals while maintaining capabilities. We view MAttr as a successful formulation of interpretability into a learnable objective that we can tackle with gradient descent, and encourage future work along these lines. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.25518 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T13:46:15.681Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).