Metraipolis › archive › dc_0694126defed
Anchoring Speech with Semantics: A Multimodal Adapter Mechanism for Automatic Speech Recognition in Low-Resource Languages — arXiv:2608.29239
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.29239
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T08:06:54.361Z
- SHA-256 of the archived text
f83471453d842e405466b23dc6337072e313d9501801d5528998ec83acfa721e- Stable URL of this page
https://metraipolis.com/documento/dc_0694126defed
How to cite
Anchoring Speech with Semantics: A Multimodal Adapter Mechanism for Automatic Speech Recognition in Low-Resource Languages — arXiv:2608.29239 — arXiv (en), https://arxiv.org/abs/2608.29239 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0694126defed (Metraipolis archive dc_0694126defed, sha256 f83471453d842e405466b23dc6337072e313d9501801d5528998ec83acfa721e)
BibTeX
@misc{dc0694126defed,
title = {Anchoring Speech with Semantics: A Multimodal Adapter Mechanism for Automatic Speech Recognition in Low-Resource Languages — arXiv:2608.29239},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.29239},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0694126defed; sha256 f83471453d842e405466b23dc6337072e313d9501801d5528998ec83acfa721e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.29239. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Low-resource ASR remains difficult because scarce transcripts provide limited supervised evidence for target-side generation. To address this gap, we propose SAMA-ASR, a lightweight adapter mechanism that augments the decoder with semantic anchors from auxiliary translations and an acoustic anchor from speech; in principle, the mechanism can be applied to similar encoder--decoder multitask speech models. Through cross-modal adaptation, SAMA-ASR conditions decoder states on translation-derived semantic embeddings and a speech embedding, combining utterance-level meaning with speech-grounded evidence before token prediction. At evaluation time, these semantic anchors can be generated automatically by an upstream speech-to-text translator rather than supplied as oracle translations. Experiments on two 30-hour datasets covering the low-resource Sinitic varieties Taiwanese Hokkien and Hakka show that SAMA-ASR improves over acoustic, prior prompt-based, and semantic-only translation-guided baselines and remains effective in practical automatic semantic-anchor settings; translator-capacity analyses show that useful semantic anchors can be produced by a compact ST model. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Sound (cs.SD); Audio and Speech Processing (eess.AS). Pagina del lavoro: https://arxiv.org/abs/2608.29239 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T07:47:52.681Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).