Metraipolis › archive › dc_3706a0d04ecd

Grounded in Sound: Reinforcement Learning with a Frozen Acoustic Judge to Curb ASR Insertion Hallucinations — arXiv:2609.14455

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.14455
License (as archived)
CC0-1.0
Archived
2026-10-16T02:15:16.991Z
SHA-256 of the archived text
3558c94e59c16dd3ebbd87b1799459ac2db32eb777a668e68c3257fe03e75c6b
Stable URL of this page
https://metraipolis.com/documento/dc_3706a0d04ecd

How to cite

Grounded in Sound: Reinforcement Learning with a Frozen Acoustic Judge to Curb ASR Insertion Hallucinations — arXiv:2609.14455 — arXiv (en), https://arxiv.org/abs/2609.14455 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3706a0d04ecd (Metraipolis archive dc_3706a0d04ecd, sha256 3558c94e59c16dd3ebbd87b1799459ac2db32eb777a668e68c3257fe03e75c6b)
BibTeX
@misc{dc3706a0d04ecd,
  title = {Grounded in Sound: Reinforcement Learning with a Frozen Acoustic Judge to Curb ASR Insertion Hallucinations — arXiv:2609.14455},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.14455},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3706a0d04ecd; sha256 3558c94e59c16dd3ebbd87b1799459ac2db32eb777a668e68c3257fe03e75c6b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.14455. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

When reinforcement learning (RL) is used for post-training automatic speech recognition (ASR), the reward almost always lives in the text space: it compares a hypothesis with the reference and never checks whether the hypothesis is supported by the audio. On highly regular speech this licenses a shortcut - guessing from a strong language prior rather than listening. Once the acoustics degrade, the shortcut runs unchecked and emits fluent but ungrounded words, i.e., insertion errors. We propose an acoustic-fidelity reward: a GRPO reward augmented with a separately pretrained, permanently frozen, non-autoregressive character-level wav2vec2-CTC acoustic judge, used strictly at training and absent at inference, where a single model decodes greedily. Trained on LibriSpeech and evaluated across a six-tier difficulty gradient including real AMI meeting speech (33,282 utterance-condition instances), the method reduces insertion errors by 28.3% on close-talking AMI-IHM and 22.3% on far-field AMI-SDM, while lowering WER on AMI-SDM from 35.89% to 34.71% and showing no detectable WER difference on the other five tiers, against a schedule-matched WER-GRPO baseline. The insertion reduction holds under a meeting-level clustered bootstrap. Four prespecified analyses support content-conditioned insertion calibration: output collapses 85-90% on unintelligible audio that preserves energy and voice activity; the gain is not recovered by the evaluated 32-best CTC rescoring configuration, yet RL internalizes it into a single greedy decoding run; and policy-only confidence yields lower insertion-AURC in all four evaluated settings. We frame this as a mechanism paper, demonstrated in one instantiation: a 7B speech LLM with a 0.3B CTC judge.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.MM).
Categorie arXiv: Sound (cs.SD) ; Multimedia (cs.MM); Audio and Speech Processing (eess.AS).
Pagina del lavoro: https://arxiv.org/abs/2609.14455 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T01:53:51.131Z dalla pagina https://arxiv.org/catchup/cs.MM/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).