Metraipolis › archive › dc_30b9db68ee39
Beyond Static Positional Encoding: Bayesian Time-aware Attention for Non-Stationary Long-term Time Series Forecasting — arXiv:2610.19603
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.19603
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T14:17:38.692Z
- SHA-256 of the archived text
8881a9e6f8fb0fe852542be3773d0ca9f127190451155faedd6b280ee686117f- Stable URL of this page
https://metraipolis.com/documento/dc_30b9db68ee39
How to cite
Beyond Static Positional Encoding: Bayesian Time-aware Attention for Non-Stationary Long-term Time Series Forecasting — arXiv:2610.19603 — arXiv (en), https://arxiv.org/abs/2610.19603 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_30b9db68ee39 (Metraipolis archive dc_30b9db68ee39, sha256 8881a9e6f8fb0fe852542be3773d0ca9f127190451155faedd6b280ee686117f)
BibTeX
@misc{dc30b9db68ee39,
title = {Beyond Static Positional Encoding: Bayesian Time-aware Attention for Non-Stationary Long-term Time Series Forecasting — arXiv:2610.19603},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.19603},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_30b9db68ee39; sha256 8881a9e6f8fb0fe852542be3773d0ca9f127190451155faedd6b280ee686117f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.19603. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Transformers have demonstrated strong performance in long-term time series forecasting (LTSF), yet modeling non-stationary temporal dependencies remains challenging. In this paper, we interpret self-attention as a Bayesian posterior over attended positions, combining a content-based prior induced by scaled dot-product similarity with a temporal-evidence likelihood induced by positional encoding. This formulation provides a probabilistic interpretation of how positional encoding shapes the temporal dependencies captured by self-attention. Building on this perspective, we derive a population-level evidence lower bound (ELBO) that characterizes the role of positional encoding in forecasting weakly stationary time series. Our analysis further highlights a limitation of static positional encoding: when temporal dependence structures vary across input windows, a fixed temporal bias must accommodate heterogeneous dynamics and can therefore be suboptimal for non-stationary forecasting. Motivated by this analysis, we propose Bayesian Time-aware Attention (BTA), which employs a mixture-of-experts (MoE) router to adaptively combine temporal kernels and generate input-conditioned, head-specific positional biases. Extensive experiments on standard LTSF benchmarks demonstrate that BTA consistently improves forecasting performance across multiple Transformer backbones and achieves competitive overall performance with modest computational overhead and stable training. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.19603 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T13:53:35.619Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).