Metraipolis › archive › dc_262fc54d6252

Progressive Memory Transformer: Memory-Aware Attention for Time-Series — arXiv:2609.31351

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.31351
License (as archived)
CC0-1.0
Archived
2026-10-08T02:13:15.221Z
SHA-256 of the archived text
b0320fdc7370bd7c0575062d2d6999acb2a7ba1f97460255cedb90867ecdc5d2
Stable URL of this page
https://metraipolis.com/documento/dc_262fc54d6252

How to cite

Progressive Memory Transformer: Memory-Aware Attention for Time-Series — arXiv:2609.31351 — arXiv (en), https://arxiv.org/abs/2609.31351 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_262fc54d6252 (Metraipolis archive dc_262fc54d6252, sha256 b0320fdc7370bd7c0575062d2d6999acb2a7ba1f97460255cedb90867ecdc5d2)
BibTeX
@misc{dc262fc54d6252,
  title = {Progressive Memory Transformer: Memory-Aware Attention for Time-Series — arXiv:2609.31351},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.31351},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_262fc54d6252; sha256 b0320fdc7370bd7c0575062d2d6999acb2a7ba1f97460255cedb90867ecdc5d2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31351. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Time-series carry structure simultaneously at multiple scales (fine-grained variation, mid-range motifs, and global properties) and downstream tasks operate at correspondingly different scales. Most existing self-supervised learning approaches supervise representations globally via instance-level contrastive losses and limited temporal neighborhood supervision, but do not explicitly exploit the structural hierarchy. We propose a learning framework that explicitly enforces a structural hierarchy across three scales independently: a local objective for token continuity, a mid-range objective for window-level motifs, and a global objective for sequence-level agreement. Realizing this framework requires the backbone to expose a representation at each scale; we introduce \textbf{Progressive Memory Transformer} (PMT), which augments a transformer with writable, window-aligned memory that exposes the mid-range scale alongside the token and sequence-level representations conventional transformers already provide. Across seven UCR/UEA/UCI classification benchmarks, a cue-retention probe, and forecasting benchmarks, PMT learns representations that probe well at the global, mid-range, and local scales---strong low-label classification (1--5\% labels), competitive forecasting performance across multiple horizons, and quantitative and qualitative evidence that memory states capture mid-range motifs.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.31351 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T01:43:06.747Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).