Metraipolis › archive › dc_cfcd6c29d148

CyFA: Linear Sequence Modeling with Relative-Time-Partitioned Memory — arXiv:2609.36259

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.36259
License (as archived)
CC0-1.0
Archived
2026-10-07T17:28:55.056Z
SHA-256 of the archived text
bfbed2f1d4afb9d507ee5e06c1bc5f49f3dd5bb6f0964e757a07852b7c7a864f
Stable URL of this page
https://metraipolis.com/documento/dc_cfcd6c29d148

How to cite

CyFA: Linear Sequence Modeling with Relative-Time-Partitioned Memory — arXiv:2609.36259 — arXiv (en), https://arxiv.org/abs/2609.36259 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cfcd6c29d148 (Metraipolis archive dc_cfcd6c29d148, sha256 bfbed2f1d4afb9d507ee5e06c1bc5f49f3dd5bb6f0964e757a07852b7c7a864f)
BibTeX
@misc{dccfcd6c29d148,
  title = {CyFA: Linear Sequence Modeling with Relative-Time-Partitioned Memory — arXiv:2609.36259},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.36259},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cfcd6c29d148; sha256 bfbed2f1d4afb9d507ee5e06c1bc5f49f3dd5bb6f0964e757a07852b7c7a864f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36259. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Linear RNNs offer linear-time sequence processing and constant-memory decoding, but their fixed-size recurrent states must accommodate all past key--value associations. Existing forgetting mechanisms and Delta Rule updates reduce interference by selectively clearing or correcting the state, yet earlier associations can still become difficult to retrieve. We introduce CyFA (Cyclic Flow Attention), a Linear RNN with relative-time-partitioned memory. At each step, a learned clock controls the cyclic transport applied jointly to the key and value states before the current key--value pair enters the age-zero slot, thereby organizing stored associations across relative-time slots. We further derive an exact change to absolute-clock coordinates that expresses CyFA as two scalar-decay linear attention recurrences and enables efficient chunk-wise training. Across 400M--1.4B pretraining experiments with matched recurrent-state sizes, CyFA improves recall-intensive performance while maintaining competitive language modeling and high computational efficiency. At 400M, CyFA outperforms KDA on FDA (42.60 vs. 26.07) while requiring only 46.7% and 48.3% of KDA's forward and backward core-operator execution times, respectively. Our code is publicly available at \href{ this https URL }{this https URL}.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.36259 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T16:59:48.136Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).