Metraipolis › archive › dc_89955ea2720e

Elastic Selective Spectral Hybrids for Train-Once, Export-Many Budgeted Inference — arXiv:2609.32486

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32486
License (as archived)
CC0-1.0
Archived
2026-10-07T22:36:37.973Z
SHA-256 of the archived text
32a72129bc510e78c64b4478bbffcf144b111421eaa4bbf67d6d2f05f4abc09d
Stable URL of this page
https://metraipolis.com/documento/dc_89955ea2720e

How to cite

Elastic Selective Spectral Hybrids for Train-Once, Export-Many Budgeted Inference — arXiv:2609.32486 — arXiv (en), https://arxiv.org/abs/2609.32486 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_89955ea2720e (Metraipolis archive dc_89955ea2720e, sha256 32a72129bc510e78c64b4478bbffcf144b111421eaa4bbf67d6d2f05f4abc09d)
BibTeX
@misc{dc89955ea2720e,
  title = {Elastic Selective Spectral Hybrids for Train-Once, Export-Many Budgeted Inference — arXiv:2609.32486},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32486},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_89955ea2720e; sha256 32a72129bc510e78c64b4478bbffcf144b111421eaa4bbf67d6d2f05f4abc09d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32486. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Deploying a language model under different computing and latency budgets calls for compact models with different quality-cost trade-offs. Towards this end, elastic spectral state space models provide ordered and temporally decomposed channels that can be truncated, but they are associated with linear time-invariant filters that cannot selectively preserve relevant past information or forget irrelevant information as the context evolves. To address this, we introduce the Elastic Selective Spectral Hybrid (ESSH), which realizes each Hankel spectral channel as an independent recurrent unit using fitted damped rotation modes. It also features an input-dependent decay and write/read gates that make temporal retention and state update input-dependent while preserving channel-wise truncation and a structured recurrence for efficient execution. ESSH combines these selective spectral mixers with sliding-window attention and jointly trains multiple capacities by reducing spectral-channel count and feed-forward width at different rates through a two-rate capacity map with full-model distillation. The resulting models support chunked parallel training and fused recurrent decoding while avoiding computation for discarded channels. At full capacity, ESSH achieves language-modeling quality comparable to similarly sized independently trained models, while smaller exports exhibit a smooth quality-cost trade-off. We validate the effectiveness of the proposed framework using language understanding, retrieval, cross-domain text, and DNA experiments, by assessing quality retention and the trade-off against independently trained and elastic baselines. At the 1.53B model configuration, fused batch-one decoding takes 1.37 ms per token on a B300, providing a 2.14-2.80x speedup over the tested Mamba-2 and Mamba-3 implementations and 3.03x over Transformer++ at matched parameter counts.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.32486 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:27.698Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).