Metraipolis › archive › dc_262fc54d6252
Progressive Memory Transformer: Memory-Aware Attention for Time-Series — arXiv:2609.31351
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.31351
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T02:13:15.221Z
- SHA-256 of the archived text
b0320fdc7370bd7c0575062d2d6999acb2a7ba1f97460255cedb90867ecdc5d2- Stable URL of this page
https://metraipolis.com/documento/dc_262fc54d6252
How to cite
Progressive Memory Transformer: Memory-Aware Attention for Time-Series — arXiv:2609.31351 — arXiv (en), https://arxiv.org/abs/2609.31351 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_262fc54d6252 (Metraipolis archive dc_262fc54d6252, sha256 b0320fdc7370bd7c0575062d2d6999acb2a7ba1f97460255cedb90867ecdc5d2)
BibTeX
@misc{dc262fc54d6252,
title = {Progressive Memory Transformer: Memory-Aware Attention for Time-Series — arXiv:2609.31351},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.31351},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_262fc54d6252; sha256 b0320fdc7370bd7c0575062d2d6999acb2a7ba1f97460255cedb90867ecdc5d2},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31351. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Time-series carry structure simultaneously at multiple scales (fine-grained variation, mid-range motifs, and global properties) and downstream tasks operate at correspondingly different scales. Most existing self-supervised learning approaches supervise representations globally via instance-level contrastive losses and limited temporal neighborhood supervision, but do not explicitly exploit the structural hierarchy. We propose a learning framework that explicitly enforces a structural hierarchy across three scales independently: a local objective for token continuity, a mid-range objective for window-level motifs, and a global objective for sequence-level agreement. Realizing this framework requires the backbone to expose a representation at each scale; we introduce \textbf{Progressive Memory Transformer} (PMT), which augments a transformer with writable, window-aligned memory that exposes the mid-range scale alongside the token and sequence-level representations conventional transformers already provide. Across seven UCR/UEA/UCI classification benchmarks, a cue-retention probe, and forecasting benchmarks, PMT learns representations that probe well at the global, mid-range, and local scales---strong low-label classification (1--5\% labels), competitive forecasting performance across multiple horizons, and quantitative and qualitative evidence that memory states capture mid-range motifs.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.31351 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T01:43:06.747Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).