Metraipolis › archive › dc_cd46cf43cab4
Positional Encoding is All You Need for Scalable Equivariance Constraint Relaxation — arXiv:2611.05041
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.05041
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T12:41:04.780Z
- SHA-256 of the archived text
33dbab8ba0cb9c9c39debbece9cf6fb8f92ff894f9eadbbad9b0a13005567d6f- Stable URL of this page
https://metraipolis.com/documento/dc_cd46cf43cab4
How to cite
Positional Encoding is All You Need for Scalable Equivariance Constraint Relaxation — arXiv:2611.05041 — arXiv (en), https://arxiv.org/abs/2611.05041 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cd46cf43cab4 (Metraipolis archive dc_cd46cf43cab4, sha256 33dbab8ba0cb9c9c39debbece9cf6fb8f92ff894f9eadbbad9b0a13005567d6f)
BibTeX
@misc{dccd46cf43cab4,
title = {Positional Encoding is All You Need for Scalable Equivariance Constraint Relaxation — arXiv:2611.05041},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.05041},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cd46cf43cab4; sha256 33dbab8ba0cb9c9c39debbece9cf6fb8f92ff894f9eadbbad9b0a13005567d6f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05041. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Equivariant neural networks incorporate known symmetries of the data, such as permutations, rotations, or translations, directly into their architecture, often improving sample efficiency and generalization. However, strict equivariance can make optimization difficult, for example, when the data only approximately satisfy the assumed symmetry, or when the equivariance constraints induce a challenging loss landscape. Recent studies have shown that relaxing equivariance during training by introducing non-equivariant dense linear layers can ease optimization in such cases and improve the performance of the resulting strictly equivariant model, after removing the relaxation layers at test time. While effective, this approach incurs a significant computational overhead, limiting its applicability in scalable architectures. In this work, we propose a more efficient constraint-relaxation method based on positional encoding (PE). Our framework injects a positional encoding with learnable scale that breaks the model's original symmetry. This positional signal is gradually annealed to zero, recovering the original equivariant model by the end of training. Empirically, we demonstrate significant improvements for translational and rotational equivariant models with only marginal compute and memory overhead. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.05041 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T12:29:47.322Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).