Metraipolis › archive › dc_47dc48c2cbbd

baseVultra: Compact Nonlinear Recurrence with Reallocated Capacity — arXiv:2610.30154

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.30154
License (as archived): CC0-1.0
Archived: 2026-10-23T15:20:23.366Z
SHA-256 of the archived text: 086ffb40af9cfddfcf36453261bc6e3c810e569f408189dc77f1780e0d9aac16
Stable URL of this page: https://metraipolis.com/documento/dc_47dc48c2cbbd

How to cite

baseVultra: Compact Nonlinear Recurrence with Reallocated Capacity — arXiv:2610.30154 — arXiv (en), https://arxiv.org/abs/2610.30154 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_47dc48c2cbbd (Metraipolis archive dc_47dc48c2cbbd, sha256 086ffb40af9cfddfcf36453261bc6e3c810e569f408189dc77f1780e0d9aac16)

BibTeX

@misc{dc47dc48c2cbbd,
  title = {baseVultra: Compact Nonlinear Recurrence with Reallocated Capacity — arXiv:2610.30154},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.30154},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_47dc48c2cbbd; sha256 086ffb40af9cfddfcf36453261bc6e3c810e569f408189dc77f1780e0d9aac16},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.30154. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Nonlinear recurrent networks remain useful for streaming sequence modeling, yet gated cell design is still dominated by LSTM and GRU. We study whether recurrent computation can be reallocated away from gate transforms and toward candidate generation. We propose baseVultra, a compact recurrent cell that concentrates the dense recurrent budget on a single H x H candidate-generation matrix, while replacing recurrent gate matrices with O(H) diagonal interactions. The candidate update combines this dense branch with two low-rank auxiliary branches, an EMA cache, and learnable output blending; the gates additionally receive lightweight second-order access to h_{t-2} through diagonal terms. On TinyShakespeare character-level language modeling, baseVultra achieves a validation loss of 1.594 in the compact one-layer 200K setting, compared with 1.604 for a parameter-matched LSTM and 1.616 for a parameter-matched GRU, while its dense recurrent matrix-vector computation is about 54% of GRU under matched hidden size for this configuration. On text8, a roughly 45x larger benchmark, baseVultra is competitive with full-gate cells (1.854 bpc vs. 1.849 for GRU and 1.837 for LSTM). On formal-language probes, the largest gains appear under constrained hidden-state capacity (0.991 vs. 0.761 GRU and 0.691 LSTM OOD2 accuracy on Counter at H=16) and vanish as hidden size grows. These results characterize where and when candidate-focused recurrent capacity allocation helps: the benefit is concentrated in compact regimes, while full-gate cells remain competitive at scale.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.30154 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-23T15:09:38.306Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).