Metraipolis › archive › dc_2c4e2247e5f5

Diagonal Equivalence in State Space Models: Characterization, Retrieval Rank, and Selective Extensions — arXiv:2611.03101

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.03101
License (as archived)
CC0-1.0
Archived
2026-11-03T11:23:04.967Z
SHA-256 of the archived text
9374d147d67103f88aa7d00caeb98d1ad1e34280dfa3a1211c1177e8e7ff752f
Stable URL of this page
https://metraipolis.com/documento/dc_2c4e2247e5f5

How to cite

Diagonal Equivalence in State Space Models: Characterization, Retrieval Rank, and Selective Extensions — arXiv:2611.03101 — arXiv (en), https://arxiv.org/abs/2611.03101 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2c4e2247e5f5 (Metraipolis archive dc_2c4e2247e5f5, sha256 9374d147d67103f88aa7d00caeb98d1ad1e34280dfa3a1211c1177e8e7ff752f)
BibTeX
@misc{dc2c4e2247e5f5,
  title = {Diagonal Equivalence in State Space Models: Characterization, Retrieval Rank, and Selective Extensions — arXiv:2611.03101},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03101},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2c4e2247e5f5; sha256 9374d147d67103f88aa7d00caeb98d1ad1e34280dfa3a1211c1177e8e7ff752f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03101. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We classify real and complex state-space models (SSMs) with linear transitions, distinguishing fixed-basis diagonal equivalence from input-output equivalence and tracking coefficient-conditioning windows under coordinate changes. We characterize diagonal equivalence through commuting transition families whose members are diagonalizable over the chosen field. With time-shared token-local matrix maps, we identify this class's state-mediated kernels with unnormalized scalar-mode linear attention with multiplicative masks. Mamba 3's multiple-input, multiple-output (MIMO) construction lifts one value per local interface; SSD-MIMO routes independent values for local retrieval rank above one. HybridMamba combines Mamba 1's per-coordinate decay, Mamba 2-style heads and Mamba 3's complex rotations and two-endpoint writes, mixing real and rotating state blocks with different decay laws. They recover Mamba 1-3 single-input, single-output (SISO) configurations as exact special cases. On five large H100 bfloat16 workloads, our fused GPU kernels improve geometric-mean training time over official Mamba 1/3 in matched SISO recoveries, with Mamba 2 near parity at SSM-core scope. Memory tradeoffs depend on the baseline and metric: saved-for-backward versus peak allocation. In small-model multi-token recall experiments, both designs outperform Mamba 1-3 on the hardest tier; higher routing ranks perform best in gated or phase-diverse configurations. In single-seed language models with about 500M parameters, both improve on Mamba 1/2 in test loss at 2B tokens, but the routing gains do not translate into lower language-model loss. Our best configurations approach Mamba 3 SISO through 10B tokens while remaining behind it and Mamba 3 MIMO in language-model loss.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Optimization and Control (math.OC); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2611.03101 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T11:02:37.670Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).