Metraipolis › archive › dc_cd52b1d8a733
Recurrence Is Not Enough: Causally Validating Multilingual SAE Translation Features in Gemma 2 and 3 — arXiv:2609.04808
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.04808
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T22:26:41.536Z
- SHA-256 of the archived text
43cac508eca14422d615409b80bd6518e213f33e498914e0e29706f8e68be5d9- Stable URL of this page
https://metraipolis.com/documento/dc_cd52b1d8a733
How to cite
Recurrence Is Not Enough: Causally Validating Multilingual SAE Translation Features in Gemma 2 and 3 — arXiv:2609.04808 — arXiv (en), https://arxiv.org/abs/2609.04808 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cd52b1d8a733 (Metraipolis archive dc_cd52b1d8a733, sha256 43cac508eca14422d615409b80bd6518e213f33e498914e0e29706f8e68be5d9)
BibTeX
@misc{dccd52b1d8a733,
title = {Recurrence Is Not Enough: Causally Validating Multilingual SAE Translation Features in Gemma 2 and 3 — arXiv:2609.04808},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.04808},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cd52b1d8a733; sha256 43cac508eca14422d615409b80bd6518e213f33e498914e0e29706f8e68be5d9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.04808. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Sparse autoencoder (SAE) features are increasingly used to explain and steer language-model behavior, but it remains unclear whether a feature found in one language context plays the same causal role when processing prompts in another language. We study this question using translation-initiation features (Wu et al., 2026). We reproduce the SAE feature discovery method from Wu et al. in Gemma 2 and extend it to multilingual settings that vary prompt language, source language, and target language. We then test whether features that recur across settings affect translation behavior by amplifying or ablating their activations during inference. We also examine whether the method can be applied to Gemma 3. In both models, we observe an identical finding: although we can find more than 20 features that activate frequently across all discovery settings, causal validation shows that nearly all have small or inconsistent effects. In contrast, one feature -- Gemma 2's (L10, 5717) and Gemma 3's (L20, 2456) -- consistently improves COMET scores when amplified and degrades them when ablated across 23 language settings. These results show that feature recurrence can overstate cross-lingual transfer, while identifying a language-agnostic translation-initiation direction in Gemma 2 and Gemma 3. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-07 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.04808 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T22:18:11.531Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).