Metraipolis › archive › dc_f03ca6dd77ce

Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learners — arXiv:2608.19863

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.19863
License (as archived)
CC0-1.0
Archived
2026-10-18T13:41:50.341Z
SHA-256 of the archived text
f89935b7503289382b8f1ad7c97ee696bc3642f298773a9aa4cea2b5d4295833
Stable URL of this page
https://metraipolis.com/documento/dc_f03ca6dd77ce

How to cite

Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learners — arXiv:2608.19863 — arXiv (en), https://arxiv.org/abs/2608.19863 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f03ca6dd77ce (Metraipolis archive dc_f03ca6dd77ce, sha256 f89935b7503289382b8f1ad7c97ee696bc3642f298773a9aa4cea2b5d4295833)
BibTeX
@misc{dcf03ca6dd77ce,
  title = {Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learners — arXiv:2608.19863},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.19863},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f03ca6dd77ce; sha256 f89935b7503289382b8f1ad7c97ee696bc3642f298773a9aa4cea2b5d4295833},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.19863. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Self-supervised learning (SSL) has driven substantial progress in audio representation learning, though existing methods have increasingly relied on elaborate pre-training recipes to reach competitive performance. A markedly different pre-training philosophy underpins the most influential progress in language modeling and, more recently, in visual representation learning: rather than train encoders as static feature extractors, models are trained to predict the next element, a discrete token or a continuous embedding, from the preceding context. Autoregressive prediction thereby provides a unified pre-training interface that transfers across modalities, compelling the model to learn the underlying data distribution. We ask whether such a simple causal paradigm can yield strong audio learners, given that audio's temporal structure makes autoregressive prediction of patch embeddings a natural fit. We introduce NAPE (Next-Audio-Patch-Embedding prediction), a self-supervised framework in which a causal Transformer predicts each next patch embedding of a log-mel spectrogram from the previous ones, using causal masking and stop-gradient as its sole training signal. The design is intentionally minimalist, avoiding reconstruction decoders, acoustic tokenizers, student-teacher setups, and auxiliary regularization losses. Across six audio and speech benchmarks, NAPE achieves state-of-the-art fine-tuning performance on several tasks, scales consistently across encoder sizes, and yields strong linear-probing results. NAPE also produces structured attention patterns without explicit supervision.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-21 in eess.AS).
Categorie arXiv: Audio and Speech Processing (eess.AS) ; Artificial Intelligence (cs.AI); Sound (cs.SD).
Pagina del lavoro: https://arxiv.org/abs/2608.19863 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T13:39:43.199Z dalla pagina https://arxiv.org/catchup/eess.AS/2026-08-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).