Metraipolis › archive › dc_b47c743789ee
Equivariant Music Transformer — arXiv:2608.03920
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.03920
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T12:19:41.268Z
- SHA-256 of the archived text
97387784145de76241a28a45535f37ae011922a2f03db461b6968033ae9b8241- Stable URL of this page
https://metraipolis.com/documento/dc_b47c743789ee
How to cite
Equivariant Music Transformer — arXiv:2608.03920 — arXiv (en), https://arxiv.org/abs/2608.03920 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b47c743789ee (Metraipolis archive dc_b47c743789ee, sha256 97387784145de76241a28a45535f37ae011922a2f03db461b6968033ae9b8241)
BibTeX
@misc{dcb47c743789ee,
title = {Equivariant Music Transformer — arXiv:2608.03920},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.03920},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b47c743789ee; sha256 97387784145de76241a28a45535f37ae011922a2f03db461b6968033ae9b8241},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.03920. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Humans recognize a musical passage even when it is shifted in time or transposed in pitch, indicating a notion of equivariance in the representation space. Our analysis, however, shows that standard music transformers map such time-shifted or pitch-transposed inputs onto uncorrelated representations: these models become progressively less equivariant as they scale in size or train longer. This suggests that in standard music transformers, additional model capacity is allocated to memorizing absolute patterns rather than capturing shared musical structures. In this paper, we propose the Equivariant Music Transformer (EMT), which enforces equivariance through self-distillation by jointly optimizing a next-token-prediction and an auxiliary equivariance regularization loss. We find that the additional equivariance loss acts as a beneficial regularizer, simultaneously improving next-token prediction and producing equivariant latent representations. Through both objective and subjective evaluations, EMT demonstrates superior equivariance and generative capability compared to data augmentation, feature engineering, and state-of-the-art (SOTA) baselines. More broadly, our findings reveal that standard language modeling methods alone do not capture music's translational symmetries, and dedicated inductive biases are required to produce better music representations. The code, weights and demos are available online. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-05 in cs.AI). Categorie arXiv: Sound (cs.SD) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.03920 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T12:05:53.100Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).