Metraipolis › archive › dc_57b0ee1eb995
HySPE: Positional Encoding via Symplectic Dual Shears — arXiv:2610.10154
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.10154
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T11:26:21.046Z
- SHA-256 of the archived text
2b8805f5d7ededdb7848b7035afaf8e28668e9965e5059c9947de8a1f79bc826- Stable URL of this page
https://metraipolis.com/documento/dc_57b0ee1eb995
How to cite
HySPE: Positional Encoding via Symplectic Dual Shears — arXiv:2610.10154 — arXiv (en), https://arxiv.org/abs/2610.10154 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_57b0ee1eb995 (Metraipolis archive dc_57b0ee1eb995, sha256 2b8805f5d7ededdb7848b7035afaf8e28668e9965e5059c9947de8a1f79bc826)
BibTeX
@misc{dc57b0ee1eb995,
title = {HySPE: Positional Encoding via Symplectic Dual Shears — arXiv:2610.10154},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.10154},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_57b0ee1eb995; sha256 2b8805f5d7ededdb7848b7035afaf8e28668e9965e5059c9947de8a1f79bc826},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.10154. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We introduce Hyperbolic Symplectic Positional Encoding (HySPE), grounding positional attention in non-compact symplectic transformations. While canonical Rotary Position Embedding (RoPE) parameterizes the compact, elliptic branch of $\Sp(2,\R)$ via rotations, HySPE operationalizes its hyperbolic branch via a damped symmetric composition of dual shears, yielding a conformally symplectic contraction with two spectral decay rates per channel pair. To eliminate the exponential representation drift inherent to naive absolute factorizations, we diagonalize the operator in its invariant eigenbasis and introduce blockwise coordinate rebasing with adaptive centered execution. This guarantees length-independent numerical bounds while matching cached RoPE forward latency (7.21\,ms on an RTX 4090). On TinyShakespeare, HySPE-UltraLong maintains an invariant perplexity of 4.810 up to $16\times$ zero-shot extrapolation ($L=4096$), whereas RoPE degrades to 131.198. Scaled to a 51M-parameter subword Transformer on WikiText-103 ($L_{\text{train}}=512$), HySPE closely matches RoPE in-domain while robustly extrapolating to length 8192, reducing tail perplexity by 83.9\% over RoPE. While these controlled experiments establish HySPE's extrapolation robustness and numerical stability, evaluating its scaling behavior on large-scale foundation models remains an important direction for future investigation.
Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.10154 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T10:24:06.349Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).