Metraipolis › archive › dc_8be2fa3509c8
Local Prototype Reconstruction for Text-Compatible Speech-to-LLM Bridge Pretraining — arXiv:2610.11159
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.11159
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:04:08.043Z
- SHA-256 of the archived text
41b53f9b82a412354335a0de103a9b438ed5d4e19243a95e5cedbcca257d4ac9- Stable URL of this page
https://metraipolis.com/documento/dc_8be2fa3509c8
How to cite
Local Prototype Reconstruction for Text-Compatible Speech-to-LLM Bridge Pretraining — arXiv:2610.11159 — arXiv (en), https://arxiv.org/abs/2610.11159 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8be2fa3509c8 (Metraipolis archive dc_8be2fa3509c8, sha256 41b53f9b82a412354335a0de103a9b438ed5d4e19243a95e5cedbcca257d4ac9)
BibTeX
@misc{dc8be2fa3509c8,
title = {Local Prototype Reconstruction for Text-Compatible Speech-to-LLM Bridge Pretraining — arXiv:2610.11159},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.11159},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8be2fa3509c8; sha256 41b53f9b82a412354335a0de103a9b438ed5d4e19243a95e5cedbcca257d4ac9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.11159. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Speech-to-LLM systems often connect a frozen speech encoder to a frozen large language model (LLM) through a small trainable bridge. The bridge is usually treated as plumbing, but it in fact defines the geometry of the speech-to-LLM interface, and the pretraining objective decides whether that interface provides a reusable initialization for downstream tasks. We study a transferable bridge through two complementary properties: global alignment with the text side, and local lexical manifold compatibility, where bridge embeddings remain close to the frozen LLM's input-embedding neighbourhoods. We make this property measurable with a fixed, head-free, timestamp-free diagnostic that applies to any objective, and show that next-word prediction (NWP) and sentence-level contrastive pretraining do not fully capture token-level lexical compatibility. We then introduce Local Prototype Reconstruction (LPR), a lightweight training-only regularizer that requires each aligned bridge token to be reconstructable from a small neighbourhood of frozen LLM token embeddings, with a hard single-prototype anchor as its limiting case. On multilingual ASR and speech translation, LPR improves transfer, with the largest gains on translation and low-resource adaptation. Crucially, our independent diagnostic correlates with downstream gains across objectives, suggesting that lexical manifold compatibility is predictive of reusability for speech-to-LLM bridges. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Sound (cs.SD). Pagina del lavoro: https://arxiv.org/abs/2610.11159 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T05:47:15.119Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).