Metraipolis › archive › dc_561a28b02071
The Automaton Underneath: The Additive Input Pathway Is a Parasitic Attractor for State Tracking in Householder Linear RNN — arXiv:2609.18966
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.18966
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T15:04:57.723Z
- SHA-256 of the archived text
3aad935a38e6d79c93d0907b51118e0546d81e28942863300304b34795fbc8c4- Stable URL of this page
https://metraipolis.com/documento/dc_561a28b02071
How to cite
The Automaton Underneath: The Additive Input Pathway Is a Parasitic Attractor for State Tracking in Householder Linear RNN — arXiv:2609.18966 — arXiv (en), https://arxiv.org/abs/2609.18966 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_561a28b02071 (Metraipolis archive dc_561a28b02071, sha256 3aad935a38e6d79c93d0907b51118e0546d81e28942863300304b34795fbc8c4)
BibTeX
@misc{dc561a28b02071,
title = {The Automaton Underneath: The Additive Input Pathway Is a Parasitic Attractor for State Tracking in Householder Linear RNN — arXiv:2609.18966},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.18966},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_561a28b02071; sha256 3aad935a38e6d79c93d0907b51118e0546d81e28942863300304b34795fbc8c4},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.18966. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Linear RNNs with input-dependent Householder-product transitions (DeltaNet/DeltaProduct-class) can provably represent hard state-tracking automata, yet trained models fail to length-generalize -- a gap recent work attributes to optimization, without a causal account. We give one, in a pre-registered, within-architecture causal ablation: the same model with one term deleted -- the additive input injection $b_t = W_b e_t$. With $b_t$, models fit length 32 and collapse out-of-distribution on parity, $S_4$, $A_5$, and non-solvable $S_5$ word problems (0.20 at position 512 on $S_5$). Without it -- input acting only through the orthogonal transitions -- the same architecture learns the exact automaton: median accuracy 1.00 at 16x the training length, at every width admitted by a representation law we state and test: the minimal number of Householder factors per token equals the maximal reflection length of the task's generators in the format-pinned representation (parity 1, $S_4$ 3, $A_5$ and $S_5$ 4); below it, nothing fits. The contrast with DeltaProduct's $S_4$/$A_5$ at $n_h{=}2$ (group-element classification, $SO(3)$ realization) shows the law is representation-relative: task format is a hidden variable in state-tracking benchmarks. Two pre-registered arms locate the mechanism. (i) Initialized at a verified-exact solution with $W_b{=}0$, Adam grows the additive path and pulls the model off the exact solution; $-b$ controls stay at 1.00. (ii) Our registered prediction that the fit routes through $b_t$ fired its kill criterion: all 49 fitting seeds retain in-domain fit under $W_b{:=}0$ -- and at law-minimal width, zeroing $W_b$ at inference restores exact generalization (parity 5/5, $S_5$ 5/5, $S_4$ 4/5, $A_5$ 4/5). The additive pathway is parasitic: it destabilizes, then conceals, a correctly learned automaton. All 202 runs pre-registered; all numbers regenerate from artifacts.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Neural and Evolutionary Computing (cs.NE).
Pagina del lavoro: https://arxiv.org/abs/2609.18966 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T14:08:19.162Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).