Metraipolis › archive › dc_db48fa8945ec

Lévy Attention: Single-Pass Predictive Uncertainty for Continuous-Time Attention — arXiv:2608.19171

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.19171
License (as archived): CC0-1.0
Archived: 2026-10-18T13:53:37.686Z
SHA-256 of the archived text: 7cf35643ad665f364ae3b7659b0978c517ddb7d2d412f6d85aa760688c122e4a
Stable URL of this page: https://metraipolis.com/documento/dc_db48fa8945ec

How to cite

Lévy Attention: Single-Pass Predictive Uncertainty for Continuous-Time Attention — arXiv:2608.19171 — arXiv (en), https://arxiv.org/abs/2608.19171 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_db48fa8945ec (Metraipolis archive dc_db48fa8945ec, sha256 7cf35643ad665f364ae3b7659b0978c517ddb7d2d412f6d85aa760688c122e4a)

BibTeX

@misc{dcdb48fa8945ec,
  title = {Lévy Attention: Single-Pass Predictive Uncertainty for Continuous-Time Attention — arXiv:2608.19171},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.19171},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_db48fa8945ec; sha256 7cf35643ad665f364ae3b7659b0978c517ddb7d2d412f6d85aa760688c122e4a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.19171. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Deep models for irregularly-sampled time series answer queries at arbitrary continuous timestamps, yet report nothing about how far each answer should be trusted. We show the attention layer itself can close that gap: with the right stochastic formulation, the pass that makes each prediction also reports, in closed form and at no extra cost, how far it should be trusted. We introduce Lévy Attention, a cross-attention operator whose output is a stochastic integral against an inhomogeneous Poisson random measure: query-key compatibilities assemble an intensity over a continuous (time x channel) index space, the measure scatters atoms under it, and the output averages an interpolated value field at those atoms. In expectation it reduces to a mollified cosine-kernel attention, so it replaces a softmax layer and trains with exact gradients. What softmax discards, the Poisson construction preserves in closed form: the evidence $\Lambda_q$ (total compatibility mass) and the disagreement $\mathrm{tr}\,\Sigma_V(q)$ (value spread). An exact variance identity makes their combination $\hat\sigma(q)=\sqrt{\mathrm{tr}\,\Sigma_V(q)\,\varphi(\Lambda_q)}$ the root-mean-square deviation of the sampled operator, emitted by the deterministic pass with no trained head. Empirically, disagreement carries the signal, while the evidence factor swings from uninformative on dense data to strongly informative on sparse. On t-PatchGNN the operator swap costs at most 5.6% accuracy against a matched control and nothing on the sparsest dataset. The free disagreement signal improves on 20-pass MC dropout across matched five-seed suites, and $\hat\sigma$ scales a calibrated Gaussian whose zero-sample CRPS beats a fifty-draw sampler; a split-conformal wrapper reaches nominal coverage at every level, and one pass ranks 3,383 unseen patients by trust in 1.4 seconds.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-20 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.19171 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T13:50:58.970Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-20?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).