Metraipolis › archive › dc_34be2bb73b9c

Distributed Learning with Selective State Space Models: Architecture-Aware Convergence Analysis — arXiv:2610.02659

Source
arXiv
Original
https://arxiv.org/abs/2610.02659
License (as archived)
CC0-1.0
Archived
2026-10-05T22:50:10.252Z
SHA-256 of the archived text
4ac4792c0ee8aab0ef044a8f2e663e1487e178f64356254748f1e8e57901d18b
Stable URL of this page
https://metraipolis.com/documento/dc_34be2bb73b9c

How to cite

Distributed Learning with Selective State Space Models: Architecture-Aware Convergence Analysis — arXiv:2610.02659 — arXiv, https://arxiv.org/abs/2610.02659 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_34be2bb73b9c (Metraipolis archive dc_34be2bb73b9c, sha256 4ac4792c0ee8aab0ef044a8f2e663e1487e178f64356254748f1e8e57901d18b)
BibTeX
@misc{dc34be2bb73b9c,
  title = {Distributed Learning with Selective State Space Models: Architecture-Aware Convergence Analysis — arXiv:2610.02659},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.02659},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_34be2bb73b9c; sha256 4ac4792c0ee8aab0ef044a8f2e663e1487e178f64356254748f1e8e57901d18b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.02659. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Modern state space models (SSMs), such as Mamba2, provide a compelling alternative to transformers by combining linear-time sequence modeling with recurrent state-space dynamics. However, the behavior of SSMs in distributed learning settings remains poorly understood. In particular, the existing standard federated learning methods are largely architecture-agnostic, and do not account for the stability, selectivity, and state-space parameterization that characterize modern selective SSMs. To address this, we derive architecture-aware gradient and smoothness bounds for single- and multi-layer selective SSMs, and convergence bounds for FedAvg and FedProx, characterizing how recurrent stability, input-dependent discretization, and state projection norms affect federated optimization. We then numerically validate the single-layer bounds on sequences generated by a teacher SSM, using a learner that follows the analyzed recurrence. We use this analysis to formulate expectations about the effects of local training and client heterogeneity, and examine these expectations by comparing nine federated learning algorithms on Mamba2 language modeling across six text domains. These experiments illustrate how SSM-specific bounds can provide a basis for interpreting the behavior of practical federated learning algorithms.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Optimization and Control (math.OC).
Pagina del lavoro: https://arxiv.org/abs/2610.02659 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-05T22:45:55.781Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).