Metraipolis › archive › dc_5e7e59cebc36

Do Spoken Language Models Hear Speech as They Read Text? Bridging Structural Gaps Between Speech and Text — arXiv:2608.22908

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.22908
License (as archived)
CC0-1.0
Archived
2026-10-18T07:45:06.012Z
SHA-256 of the archived text
dcfa57eeba944c90ddb3573f9355ca81f9c8456877f395d64b1a06adc9346c23
Stable URL of this page
https://metraipolis.com/documento/dc_5e7e59cebc36

How to cite

Do Spoken Language Models Hear Speech as They Read Text? Bridging Structural Gaps Between Speech and Text — arXiv:2608.22908 — arXiv (en), https://arxiv.org/abs/2608.22908 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5e7e59cebc36 (Metraipolis archive dc_5e7e59cebc36, sha256 dcfa57eeba944c90ddb3573f9355ca81f9c8456877f395d64b1a06adc9346c23)
BibTeX
@misc{dc5e7e59cebc36,
  title = {Do Spoken Language Models Hear Speech as They Read Text? Bridging Structural Gaps Between Speech and Text — arXiv:2608.22908},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.22908},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5e7e59cebc36; sha256 dcfa57eeba944c90ddb3573f9355ca81f9c8456877f395d64b1a06adc9346c23},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.22908. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Spoken Language Models (SLMs) generate textual responses directly from speech, offering an alternative to cascaded systems. Despite recent advances, existing SLMs still exhibit weaker instruction-following behavior and limited generalization across diverse tasks compared to text-based language models. Our analysis shows that speech and text representations in current SLMs remain weakly aligned despite strong downstream performance, indicating that structural differences between continuous, temporally varying speech and discrete text remain insufficiently addressed. To address this, we propose a simple framework that decouples length mismatch from semantic alignment and encourages closer correspondence between speech and text representations. Experiments across multiple benchmarks demonstrate competitive performance against strong baselines, underscoring the importance of explicitly addressing structural differences between speech and text in SLM training. Our code is publicly available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-25 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.22908 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T07:37:52.906Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).