Metraipolis › archive › dc_7a2dba9b138d

Where Should a Gate Look? Input vs. State Gating in Diagonal State Space Models — arXiv:2611.11336

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.11336
License (as archived)
CC0-1.0
Archived
2026-11-10T18:43:19.521Z
SHA-256 of the archived text
80b8dddb45184eccc0ee51df04283885cae2a1bd65cc48cbfe507e26db7883fb
Stable URL of this page
https://metraipolis.com/documento/dc_7a2dba9b138d

How to cite

Where Should a Gate Look? Input vs. State Gating in Diagonal State Space Models — arXiv:2611.11336 — arXiv (en), https://arxiv.org/abs/2611.11336 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7a2dba9b138d (Metraipolis archive dc_7a2dba9b138d, sha256 80b8dddb45184eccc0ee51df04283885cae2a1bd65cc48cbfe507e26db7883fb)
BibTeX
@misc{dc7a2dba9b138d,
  title = {Where Should a Gate Look? Input vs. State Gating in Diagonal State Space Models — arXiv:2611.11336},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.11336},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7a2dba9b138d; sha256 80b8dddb45184eccc0ee51df04283885cae2a1bd65cc48cbfe507e26db7883fb},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.11336. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Modern state-space models owe much of their computational efficiency to their linear, often diagonal, dynamics. Classical recurrent neural networks instead allow nonlinear state-to-state interactions, which can be beneficial when memory updates need to depend on the past context. In this paper, we ask whether state feedback through a simple gate regulating memory updates can enhance the expressivity of diagonal state-space models. To investigate this question, we compare gates that depend on the current input, the recurrent state, or both. We establish a strict expressivity separation: for scalar models, state gating solves induction-head tasks for arbitrary vocabulary size and sequence length, whereas input gating fails on a finite instance under standard stability assumptions on the state transition matrix. Experiments show that state gating retains its advantage on induction-head tasks in higher-dimensional models, whereas input gating performs better on selective copying tasks. Together, our results show that even a simple, diagonal form of state-feedback gating can unlock capabilities unavailable to input-controlled models, while input and state gating offer complementary strengths and achieve the strongest overall performance when combined.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.11336 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.636Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).