Metraipolis › archive › dc_cd52b1d8a733

Recurrence Is Not Enough: Causally Validating Multilingual SAE Translation Features in Gemma 2 and 3 — arXiv:2609.04808

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.04808
License (as archived)
CC0-1.0
Archived
2026-10-16T22:26:41.536Z
SHA-256 of the archived text
43cac508eca14422d615409b80bd6518e213f33e498914e0e29706f8e68be5d9
Stable URL of this page
https://metraipolis.com/documento/dc_cd52b1d8a733

How to cite

Recurrence Is Not Enough: Causally Validating Multilingual SAE Translation Features in Gemma 2 and 3 — arXiv:2609.04808 — arXiv (en), https://arxiv.org/abs/2609.04808 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cd52b1d8a733 (Metraipolis archive dc_cd52b1d8a733, sha256 43cac508eca14422d615409b80bd6518e213f33e498914e0e29706f8e68be5d9)
BibTeX
@misc{dccd52b1d8a733,
  title = {Recurrence Is Not Enough: Causally Validating Multilingual SAE Translation Features in Gemma 2 and 3 — arXiv:2609.04808},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.04808},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cd52b1d8a733; sha256 43cac508eca14422d615409b80bd6518e213f33e498914e0e29706f8e68be5d9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.04808. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Sparse autoencoder (SAE) features are increasingly used to explain and steer language-model behavior, but it remains unclear whether a feature found in one language context plays the same causal role when processing prompts in another language. We study this question using translation-initiation features (Wu et al., 2026). We reproduce the SAE feature discovery method from Wu et al. in Gemma 2 and extend it to multilingual settings that vary prompt language, source language, and target language. We then test whether features that recur across settings affect translation behavior by amplifying or ablating their activations during inference. We also examine whether the method can be applied to Gemma 3. In both models, we observe an identical finding: although we can find more than 20 features that activate frequently across all discovery settings, causal validation shows that nearly all have small or inconsistent effects. In contrast, one feature -- Gemma 2's (L10, 5717) and Gemma 3's (L20, 2456) -- consistently improves COMET scores when amplified and degrades them when ablated across 23 language settings. These results show that feature recurrence can overstate cross-lingual transfer, while identifying a language-agnostic translation-initiation direction in Gemma 2 and Gemma 3.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-07 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.04808 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T22:18:11.531Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).