Metraipolis › archive › dc_28e9bda30dfd

Voltic: Distinguishing Volatility from Stochasticity in Recurrent Memory — arXiv:2610.05700

Source: arXiv
Original: https://arxiv.org/abs/2610.05700
License (as archived): CC0-1.0
Archived: 2026-10-06T05:28:53.259Z
SHA-256 of the archived text: 2737691e0e1f8b911f0178346ea312cdbc1e85efc77a0626b340ca63d02f6275
Stable URL of this page: https://metraipolis.com/documento/dc_28e9bda30dfd

How to cite

Voltic: Distinguishing Volatility from Stochasticity in Recurrent Memory — arXiv:2610.05700 — arXiv, https://arxiv.org/abs/2610.05700 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_28e9bda30dfd (Metraipolis archive dc_28e9bda30dfd, sha256 2737691e0e1f8b911f0178346ea312cdbc1e85efc77a0626b340ca63d02f6275)

BibTeX

@misc{dc28e9bda30dfd,
  title = {Voltic: Distinguishing Volatility from Stochasticity in Recurrent Memory — arXiv:2610.05700},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.05700},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_28e9bda30dfd; sha256 2737691e0e1f8b911f0178346ea312cdbc1e85efc77a0626b340ca63d02f6275},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05700. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recurrent sequence models must decide how strongly to overwrite their memory at each token. Read as Bayesian filtering, this write is the gain of a Kalman update, set by uncertainty from two sources that pull it in opposite directions: volatility, how quickly the underlying associations change, and stochasticity, how noisy each observation of them is. First, we show that the update of gated delta-rule memories is the form this filter takes under isotropic uncertainty. Next, we introduce Voltic, a recurrent memory that keeps the covariance anisotropic and makes both noise variances input-dependent, so the write is vector-valued and carries uncertainty accumulated over the sequence. A dense covariance would have to be propagated token by token, ruling out the parallel training these models depend on. We therefore give two assumed-density approximations, diagonal and quasi-diagonal, both of which leave the memory update in delta-rule form and reuse its chunked kernels. On controlled recall tasks in which associations change and observations are corrupted, Voltic leads all baselines. On the task combining volatility and stochasticity, its margin over the strongest baseline is larger at both extrapolation sizes than at the training sizes. In 45M-parameter language models it leads an eight-task reasoning average and achieves higher retrieval accuracy beyond the training context length than gated baselines, at throughput close to those baselines. Deriving the write from an uncertainty recursion therefore makes memory more responsive to change.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.05700 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:20.448Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).